[readline] Multibyte invisible chars cause weird prompt length calculation issue

[Алексей Шилин]

Configuration Information:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -g -O2 -fdebug-prefix-map=/build/bash-REtn6r/bash-
5.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wall
-Wno-parentheses -Wno-format-security
uname output: Linux sid-readline 4.19.0-6-amd64 #1 SMP Debian 4.19.67-
2+deb10u2 (2019-11-11) x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 5.0
Patch Level: 11
Release Status: release

Description:

I'm using the following PS1 prompt (Debian's default for normal users):

\[\e]0;\u@\h:
\w\a\]${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\
]:\[\033[01;34m\]\w\[\033[00m\]\$ 

...where the first block '\[\e]0;\u@\h: \w\a\]' is for setting xterm's
title, and the rest is Debian's "fancy" shell prompt.

The issue is, when \w contains multibyte characters (e.g. Cyrillic),
some weird prompt length calculation issues start happening (see
below).

Repeat-By:

test@sid-readline:~$ PS1='\[\e]0;\u@\h:
\w\a\]${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\
]:\[\033[01;34m\]\w\[\033[00m\]\$ '
test@sid-readline:~$ mkdir -p тестовый/каталог
test@sid-readline:~$ cd тестовый/каталог/
test@sid-readline:~/тестовый/каталог$ echo hello cruel world
hello cruel world
test@sid-readline:~/тестовый/каталог$ 

# Press C-r for history backward search:

(reverse-i-search)`':стовый/каталог$ 

                      ^----- cursor is here

# Note that part of the previous prompt is still visible.
# Enter 'he':

(reverse-i-search)`he': echo hello cruel world

# ...and press [ End ]:

test@sid-readline:~/тестовый/каталог$ l world               

                                        cursor is here -----^

# As you see, the command line is shifted to the left by the same 
# number of chars as the amount of "extra" bytes in \w.

# Now let's delete \w from the *first* ("xterm's") invisible 
# characters block:

test@sid-readline:~/тестовый/каталог$ PS1='\[\e]0;\u@\h:
\a\]${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:
\[\033[01;34m\]\w\[\033[00m\]\$ '
test@sid-readline:~/тестовый/каталог$ 

# Press C-r again:

(reverse-i-search)`': 

# Now there's no junk left in the prompt line.
# Enter 'he' again and press [ End ]:

test@sid-readline:~/тестовый/каталог$ echo hello cruel world

# As you see, now everything works as expected i.e. the issue seems
# to be in the way readline is handling multibyte invisible characters 
# in a prompt, *probably* in the rl_redisplay() function.