bug#32345: 12.1.1; wrong preview position for four-byte utf-8 encoded character (xelatex)

[sattler . christian]

The attached log is generated by previewing the following document with xelatex:

\documentclass{article}

\begin{document}

Test: $𝒞$

\end{document}

The unicode character in math mode after "Test: " is U+1D49E.
The preview range for the snippet is completely wrong and includes the
entire line before the snipped. It appears that there are some encoding
problems because the log file refers to a unicode character U+D49E,
which is the lowest 16 bits of the original character's codepoint.

Emacs  : GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.30)
 of 2018-07-05
Package: 12.1.1

Run buffer contents:

Running `Preview-LaTeX' on `test' with ``xelatex  --synctex=1 -file-line-error 
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input" test.tex''
This is XeTeX, Version 3.14159265-2.6-0.99999 (TeX Live 2018/Arch Linux) 
(preloaded format=xelatex)
 restricted \write18 enabled.
entering extended mode
LaTeX2e <2018-04-01> patch level 2
Babel <3.18> and hyphenation patterns for 84 language(s) loaded.

No auxiliary output files.

(./test.tex (/usr/share/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/share/texmf-dist/tex/latex/base/size10.clo)) (./test.aux) 
(/home/noname/.emacs.d/elpa/auctex-12.1.1/latex/preview.sty 
(/usr/share/texmf-dist/tex/generic/luatex85/luatex85.sty) 
(/home/noname/.emacs.d/elpa/auctex-12.1.1/latex/prtightpage.def) 
(/home/noname/.emacs.d/elpa/auctex-12.1.1/latex/prauctex.def
No auxiliary output files.


(/home/noname/.emacs.d/elpa/auctex-12.1.1/latex/prauctex.cfg)) 
(/home/noname/.emacs.d/elpa/auctex-12.1.1/latex/prfootnotes.def)
Preview: Fontsize 10pt
Preview: PDFoutput 1
)
./test.tex:5: Preview: Snippet 1 started.
<-><->
      
l.5 Test: $
           풞$
Preview: Tightpage -32891 -32891 32891 32891
./test.tex:5: Preview: Snippet 1 ended.(0+0x0).
<-><->
      
l.5 Test: $풞$
             
[1] )
(see the transcript file for additional information)
Output written on test.pdf (1 page).
SyncTeX written on test.synctex.gz.
Transcript written on test.log.

TeX Output exited as expected with code 1 at Thu Aug  2 01:59:30
Running `Preview-PDF2DSC' with ``pdf2dsc test.pdf 
test.prv/tmp2pIiJl/preview.dsc''

Preview-PDF2DSC finished at Thu Aug  2 01:59:30
Running `Preview-Ghostscript' with ``/usr/bin/rungs 
-dOutputFile\=\(test.prv/tmp2pIiJl/pr1-\%d.png\) -q -dDELAYSAFER -dNOPAUSE 
-DNOPLATFONTS -dPrinted -dTextAlphaBits\=4 -dGraphicsAlphaBits\=4 
-sDEVICE\=png16m -r94.08x93.9321''

Preview-Ghostscript finished at Thu Aug  2 01:59:30


current state:
==============
(setq
 AUCTeX-version "12.1.1"
 LaTeX-command-style '(("" "%(PDF)%(latex) %(extraopts) %S%(PDFout) 
-file-line-error"))
 image-types '(svg imagemagick png gif tiff jpeg xpm xbm pbm postscript)
 preview-image-type 'png
 preview-image-creators '((dvipng (open preview-gs-open 
preview-dvipng-process-setup) (place preview-gs-place)
                           (close preview-dvipng-close))
                          (png (open preview-gs-open) (place preview-gs-place) 
(close preview-gs-close))
                          (jpeg (open preview-gs-open) (place preview-gs-place) 
(close preview-gs-close))
                          (pnm (open preview-gs-open) (place preview-gs-place) 
(close preview-gs-close))
                          (tiff (open preview-gs-open) (place preview-gs-place) 
(close preview-gs-close)))
 preview-dvipng-image-type 'png
 preview-dvipng-command "dvipng -picky -noghostscript %d -o 
\"%m/prev%%03d.png\""
 preview-pdf2dsc-command "pdf2dsc %s.pdf %m/preview.dsc"
 preview-gs-command "/usr/bin/rungs"
 preview-gs-options '("-q" "-dDELAYSAFER" "-dNOPAUSE" "-DNOPLATFONTS" 
"-dPrinted" "-dTextAlphaBits=4"
                      "-dGraphicsAlphaBits=4")
 preview-gs-image-type-alist '((png png "-sDEVICE=png16m") (dvipng png 
"-sDEVICE=png16m")
                               (jpeg jpeg "-sDEVICE=jpeg") (pnm pbm 
"-sDEVICE=pnmraw")
                               (tiff tiff "-sDEVICE=tiff12nc"))
 preview-fast-conversion t
 preview-prefer-TeX-bb nil
 preview-dvips-command "dvips -Pwww -i -E %d -o %m/preview.000"
 preview-fast-dvips-command "dvips -Pwww %d -o %m/preview.ps"
 preview-scale-function 'preview-scale-from-face
 preview-LaTeX-command '("%`%l \"\\nonstopmode\\nofiles\\PassOptionsToPackage{"
                         ("," . preview-required-option-list)
                         
"}{preview}\\AtBeginDocument{\\ifx\\ifPreview\\undefined" 
preview-default-preamble
                         "\\fi}\"%' %t")
 preview-required-option-list '("active" "tightpage" "auctex" 
(preview-preserve-counters "counters"))
 preview-preserve-counters nil
 preview-default-option-list '("displaymath" "floats" "graphics" "textmath" 
"sections" "footnotes")
 preview-default-preamble '("\\RequirePackage[" ("," . 
preview-default-option-list) "]{preview}[2004/11/05]")
 preview-LaTeX-command-replacements nil
 preview-dump-replacements '(preview-LaTeX-command-replacements
                             ("\\`\\([^ ]+\\)\\(\\( +-\\([^ 
\\\\\"]\\|\\\\\\.\\|\"[^\"]*\"\\)*\\)*\\)\\(.*\\)\\'"
                              "\\1 -ini -interaction=nonstopmode \"&\\1\" " 
preview-format-name ".ini \\5")
                             )
 preview-undump-replacements '(("\\`\\([^ ]+\\) .*? \"\\\\input\" \\(.*\\)\\'" 
"\\1 -interaction=nonstopmode \"&"
                                preview-format-name "\" \\2")
                               )
 preview-auto-cache-preamble 'ask
 preview-TeX-style-dir "/home/noname/.emacs.d/elpa/auctex-12.1.1/latex"
 )

Output from running `/usr/bin/rungs -h':
GPL Ghostscript 9.23 (2018-03-21)
Copyright (C) 2018 Artifex Software, Inc.  All rights reserved.
Usage: gs [switches] [file1.ps file2.ps ...]
Most frequently used switches: (you can use # in place of =)
 -dNOPAUSE           no pause after page   | -q       `quiet', fewer messages
 -g<width>x<height>  page size in pixels   | -r<res>  pixels/inch resolution
 -sDEVICE=<devname>  select device         | -dBATCH  exit after last file
 -sOutputFile=<file> select output file: - for stdout, |command for pipe,
                                         embed %d or %ld for page #
Input formats: PostScript PostScriptLevel1 PostScriptLevel2 PostScriptLevel3 PDF
Default output device: x11alpha
Available devices:
   alc1900 alc2000 alc4000 alc4100 alc8500 alc8600 alc9100 ap3250 atx23
   atx24 atx38 bbox bit bitcmyk bitrgb bitrgbtags bj10e bj10v bj10vh bj200
   bjc600 bjc800 bjc880j bjccmyk bjccolor bjcgray bjcmono bmp16 bmp16m
   bmp256 bmp32b bmpgray bmpmono bmpsep1 bmpsep8 ccr cdeskjet cdj1600 cdj500
   cdj550 cdj670 cdj850 cdj880 cdj890 cdj970 cdjcolor cdjmono cdnj500 cfax
   chp2200 cif cljet5 cljet5c cljet5pr coslw2p coslwxl cups declj250 deskjet
   devicen dfaxhigh dfaxlow display dj505j djet500 djet500c dl2100 dnj650c
   epl2050 epl2050p epl2120 epl2500 epl2750 epl5800 epl5900 epl6100 epl6200
   eplcolor eplmono eps2write eps9high eps9mid epson epsonc escp escpage
   faxg3 faxg32d faxg4 fmlbp fmpr fpng fs600 gdi gprf hl1240 hl1250 hl7x0
   hpdj1120c hpdj310 hpdj320 hpdj340 hpdj400 hpdj500 hpdj500c hpdj510
   hpdj520 hpdj540 hpdj550c hpdj560c hpdj600 hpdj660c hpdj670c hpdj680c
   hpdj690c hpdj850c hpdj855c hpdj870c hpdj890c hpdjplus hpdjportable ibmpro
   ijs imagen inferno ink_cov inkcov itk24i itk38 jetp3852 jj100 jpeg
   jpegcmyk jpeggray la50 la70 la75 la75plus laserjet lbp310 lbp320 lbp8
   lex2050 lex3200 lex5700 lex7000 lips2p lips3 lips4 lips4v lj250 lj3100sw
   lj4dith lj4dithp lj5gray lj5mono ljet2p ljet3 ljet3d ljet4 ljet4d
   ljet4pjl ljetplus ln03 lp1800 lp1900 lp2000 lp2200 lp2400 lp2500 lp2563
   lp3000c lp7500 lp7700 lp7900 lp8000 lp8000c lp8100 lp8200c lp8300c
   lp8300f lp8400f lp8500c lp8600 lp8600f lp8700 lp8800c lp8900 lp9000b
   lp9000c lp9100 lp9200b lp9200c lp9300 lp9400 lp9500c lp9600 lp9600s
   lp9800c lps4500 lps6500 lq850 lxm3200 lxm5700m m8510 md1xMono md2k
   md50Eco md50Mono md5k mgr4 mgr8 mgrgray2 mgrgray4 mgrgray8 mgrmono miff24
   mj500c mj6000c mj700v2c mj8000c ml600 necp6 npdl nullpage oce9050 oki182
   oki4w okiibm oprp opvp paintjet pam pamcmyk32 pamcmyk4 pbm pbmraw pcl3
   pclm pcx16 pcx24b pcx256 pcxcmyk pcxgray pcxmono pdfimage24 pdfimage32
   pdfimage8 pdfwrite pdfwrite pdfwrite pgm pgmraw pgnm pgnmraw photoex
   picty180 pj pjetxl pjxl pjxl300 pkm pkmraw pksm pksmraw plan plan9bm
   planc plang plank planm plib plibc plibg plibk plibm png16 png16m png256
   png48 pngalpha pnggray pngmono pngmonod pnm pnmraw ppm ppmraw pr1000
   pr1000_4 pr150 pr201 ps2write psdcmyk psdcmykog psdrgb pwgraster pxlcolor
   pxlmono r4081 rinkj rpdl samsunggdi sj48 spotcmyk st800 stcolor t4693d2
   t4693d4 t4693d8 tek4696 tiff12nc tiff24nc tiff32nc tiff48nc tiff64nc
   tiffcrle tiffg3 tiffg32d tiffg4 tiffgray tifflzw tiffpack tiffscaled
   tiffscaled24 tiffscaled32 tiffscaled4 tiffscaled8 tiffsep tiffsep1
   txtwrite uniprint x11 x11alpha x11cmyk x11cmyk2 x11cmyk4 x11cmyk8
   x11gray2 x11gray4 x11mono xcf xes xpswrite
Search path:
   /usr/share/ghostscript/9.23/Resource/Init :
   /usr/share/ghostscript/9.23/lib :
   /usr/share/ghostscript/9.23/Resource/Font :
   /usr/share/ghostscript/fonts : /usr/share/fonts/gsfonts
Ghostscript is also using fontconfig to search for font files
For more information, see /usr/share/doc/ghostscript/9.23/Use.htm.
Please report bugs to bugs.ghostscript.com.