bug#33139: 12.1.1; Running syntax-ppss breaks font-latex

[Frank Dai]

Hello AUCTeX maintainers,

Currently using smartparens with AUCTex will break font-latex non
deterministically. Attached is a screenshot of what it looks like when
broken. After some debugging, I concluded that the issue with
smartparens was that a 'yntax-ppss call was put into pre-command-hook.
The issue is reproductible with the following .emacs file (plus the
necessary AUCTeX importing) as well as an attached .tex:
(defun call-syntax-ppss ()
  (syntax-ppss))
(add-hook 'pre-command-hook 'call-syntax-ppss)

syntax-ppss should be stateless, so calling it should not affect
font-latex at all. The relevant changes that syntax-ppss makes is
modification of syntax-ppss-narrow (through syntax-ppss--data) in
lines 609-610 (in emacs-26.1) of syntax.el. I once found a case where
setting syntax-ppss--directly broke fontification, without even
calling syntax-ppss at all (but as it is nondeterministic, it no
longer works).

I have found a workaround, by putting the following line in .emacs
(add-hook 'LaTeX-mode-hook (lambda () (add-to-list
'font-latex-extend-region-functions  (lambda (min max) (point-min)))))
The idea behind this function was just to always extend the region to
the beginning of the buffer when fontifying. It seems to work, even
with syntax-ppss in 'pre-command-hook. From some cursory debugging,
none of the current functions in 'font-latex-extend-region-functions
check for presence of an unclosed $ or $$, which make it seem to me
that merely adding a function to check for an unclosed $ or $$ (or
modifying an existing one to do so) would fix this issue.

Thanks,
Frank Dai

Screenshot from 2018-10-23 23-38-30.png
Description: PNG image

hw.tex
Description: Text Data