Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 8 additions & 15 deletions Doc/library/curses.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1333,9 +1333,6 @@ Window objects
and the color pair is not limited to the value that fits in a
:func:`color_pair`.

This method is only available if Python was built against a wide-character
version of the underlying curses library.

.. versionadded:: next


Expand Down Expand Up @@ -1483,9 +1480,6 @@ Window objects
followed by combining characters) together with its attributes and color
pair, none of which :meth:`inch` can represent.

This method is only available if Python was built against a wide-character
version of the underlying curses library.

.. versionadded:: next


Expand Down Expand Up @@ -1585,9 +1579,6 @@ Window objects
The result can be written back unchanged with :meth:`addstr` (a read and a
re-write is a round-trip that preserves every cell's rendition).

This method is only available if Python was built against a wide-character
version of the underlying curses library.

.. versionadded:: next


Expand Down Expand Up @@ -2004,7 +1995,7 @@ Complex character objects
.. class:: complexchar(text, /, attr=0, pair=0)

A *complex character* (or *complexchar*) is an immutable styled
wide-character cell: a spacing character optionally followed by combining
character cell: a spacing character optionally followed by combining
characters, together with a set of attributes and a color pair.

*text* is the cell's text, *attr* a combination of the
Expand All @@ -2025,8 +2016,10 @@ Complex character objects
:func:`str` returns the cell's text; two complex characters are equal when
their text, attributes and color pair all match.

This type is only available if Python was built against a wide-character
version of the underlying curses library.
The same code works on both wide- and narrow-character builds. On a narrow
build a cell holds a single character (no combining marks) that must encode to
one byte in the window's encoding (8-bit locales only), and *pair* is limited
to the value that fits in a :func:`color_pair`.

.. attribute:: attr

Expand All @@ -2042,7 +2035,7 @@ Complex character objects
.. class:: complexstr(cells[, attr[, pair]])

A *complex character string* (or *complexstr*) is an immutable sequence of
styled wide-character cells -- the string counterpart of
styled character cells -- the string counterpart of
:class:`complexchar` (as :class:`str` is to a single character).

If *cells* is a string, it is split into character cells (each a spacing
Expand Down Expand Up @@ -2070,8 +2063,8 @@ Complex character objects
:class:`complexchar` (or strings); a :class:`!complexstr` is the immutable
form returned by a read.

This type is only available if Python was built against a wide-character
version of the underlying curses library.
Like :class:`complexchar`, this type works on both wide- and narrow-character
builds, with the same per-cell limitations on a narrow build.

.. versionadded:: next

Expand Down
21 changes: 15 additions & 6 deletions Doc/whatsnew/3.16.rst
Original file line number Diff line number Diff line change
Expand Up @@ -111,8 +111,12 @@ curses
and the module functions :func:`curses.erasewchar`, :func:`curses.killwchar`
and :func:`curses.wunctrl`, the wide-character counterparts of
:func:`curses.erasechar`, :func:`curses.killchar` and :func:`curses.unctrl`.
These features are only available when built against the wide-character
ncursesw library.
On a narrow (non-ncursesw) build the character cell holds a single character
without combining marks, representable as one byte in the window's encoding,
and :meth:`~curses.window.in_wstr` returns its decoded text;
:meth:`~curses.window.get_wstr` and the :func:`curses.erasewchar`,
:func:`curses.killwchar` and :func:`curses.wunctrl` functions require the
wide-character ncursesw library.
(Contributed by Serhiy Storchaka in :gh:`151757`.)

* Add :func:`curses.nofilter`, which undoes the effect of :func:`curses.filter`.
Expand All @@ -139,21 +143,26 @@ curses
(Contributed by Serhiy Storchaka in :gh:`152219`.)

* Add the :class:`curses.complexchar` type, representing a styled
wide-character cell (its text, attributes and color pair), and the window
character cell (its text, attributes and color pair), and the window
methods :meth:`~curses.window.in_wch` and :meth:`~curses.window.getbkgrnd`
that return one --- the wide-character counterparts of
that return one --- the counterparts of
:meth:`~curses.window.inch` and :meth:`~curses.window.getbkgd`. The
character-cell methods, such as :meth:`~curses.window.addch` and
:meth:`~curses.window.border`, now also accept a
:class:`~curses.complexchar`.
:class:`~curses.complexchar`. These work whether or not Python was built
against a wide-character-aware curses library; on a narrow build a cell holds a
single character representable as one byte in the window's encoding (so only
8-bit locales are supported).
(Contributed by Serhiy Storchaka in :gh:`152233`.)

* Add the :class:`curses.complexstr` type, an immutable run of styled cells
(the string counterpart of :class:`~curses.complexchar`), and the window
method :meth:`~curses.window.in_wchstr` that returns one. The string-cell
methods :meth:`~curses.window.addstr`, :meth:`~curses.window.addnstr`,
:meth:`~curses.window.insstr` and :meth:`~curses.window.insnstr` now also
accept a :class:`~curses.complexstr`.
accept a :class:`~curses.complexstr`. Like :class:`~curses.complexchar`, it
works whether or not Python was built against a wide-character-aware curses
library.
(Contributed by Serhiy Storchaka in :gh:`152233`.)

* Add the :mod:`curses` window method :meth:`~curses.window.dupwin`, which
Expand Down
44 changes: 37 additions & 7 deletions Lib/curses/textpad.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,13 +56,42 @@ def _update_max_yx(self):
self.maxy = maxy - 1
self.maxx = maxx - 1

def _decode(self, ch):
# The text of a chtype cell or input byte, decoded with the window's
# encoding. A_CHARTEXT keeps the character byte, dropping the attributes.
return bytes([ch & curses.A_CHARTEXT]).decode(self.win.encoding, 'replace')

def _char_at(self, *yx):
# The text of the cell at the given position (default: the cursor).
# instr() re-encodes it to the window's encoding; inch() cannot
# represent a non-ASCII 8-bit-locale character on a wide build.
return self.win.instr(*yx, 1).decode(self.win.encoding, 'replace')

def _cell_at(self, *yx):
# The cell at the given position (default: the cursor) as a chtype
# addch() can write back with its rendition. inch() mangles a non-ASCII
# character on a wide build, so take the byte from instr() and the
# attributes from inch().
return self.win.instr(*yx, 1)[0] | self.win.inch(*yx) & curses.A_ATTRIBUTES

def _isprint(self, cell):
# Whether a chtype cell holds a printable character; _decode() drops the
# attribute bits.
return self._decode(cell).isprintable()

def _printable_key(self, ch):
# Whether the integer keystroke is a printable character, not a key
# code. 0..255 are character bytes (decoded with the window's encoding);
# larger values are function and navigation keys.
return ch <= 0xff and self._decode(ch).isprintable()

def _end_of_line(self, y):
"""Go to the location of the first blank on the given line,
returning the index of the last non-blank character."""
self._update_max_yx()
last = self.maxx
while True:
if curses.ascii.ascii(self.win.inch(y, last)) != curses.ascii.SP:
if self._char_at(y, last) != ' ':
last = min(self.maxx, last+1)
break
elif last == 0:
Expand All @@ -76,15 +105,16 @@ def _insert_printable_char(self, ch):
backyx = None
while True:
if self.insert_mode:
oldch = self.win.inch()
oldch = self._cell_at()
if y >= self.maxy and x >= self.maxx:
# Use insch() in the lower-right cell: addch() there would move
# the cursor out of the window, raising an error and scrolling
# a scrollable window.
self.win.insch(ch)
# a scrollable window. Pass it as text: insch() does not decode
# an int byte through the locale on a wide build.
self.win.insch(self._decode(ch), ch & curses.A_ATTRIBUTES)
break
self.win.addch(ch)
if not self.insert_mode or not curses.ascii.isprint(oldch):
if not self.insert_mode or not self._isprint(oldch):
break
ch = oldch
(y, x) = self.win.getyx()
Expand All @@ -100,7 +130,7 @@ def do_command(self, ch):
self._update_max_yx()
(y, x) = self.win.getyx()
self.lastcmd = ch
if curses.ascii.isprint(ch):
if self._printable_key(ch):
self._insert_printable_char(ch)
elif ch == curses.ascii.SOH: # ^a
self.win.move(y, 0)
Expand Down Expand Up @@ -174,7 +204,7 @@ def gather(self):
for x in range(self.maxx+1):
if self.stripspaces and x > stop:
break
result = result + chr(curses.ascii.ascii(self.win.inch(y, x)))
result = result + self._char_at(y, x)
if self.maxy > 0:
result = result + "\n"
return result
Expand Down
97 changes: 90 additions & 7 deletions Lib/test/test_curses.py
Original file line number Diff line number Diff line change
Expand Up @@ -313,6 +313,8 @@ def test_refresh_control(self):
# 'é' common to the Latin encodings
# '¤'/'€'/'є' byte 0xA4 in ISO-8859-1 / ISO-8859-15 / KOI8-U
# Precomposed characters are used so a round-trip does not depend on the form.
# On a narrow (non-wide) build a cell holds one byte, so cases that need a
# combining sequence or a multibyte character are guarded with _storable().

def _encodable(self, s):
# Wide characters are only supported in a locale that can encode them.
Expand All @@ -322,6 +324,18 @@ def _encodable(self, s):
return False
return True

def _storable(self, s):
# Text the current build can place in character cells. A wide build
# stores any locale-encodable text (combining sequences and multibyte
# characters included). A narrow build has no wide-character cells, so
# each character must occupy a single cell -- that is, encode to exactly
# one byte.
if not self._encodable(s):
return False
if hasattr(self.stdscr, 'get_wch'): # wide build
return True
return len(s.encode(self.stdscr.encoding)) == len(s)

def _read_char(self, y, x):
# The character written to a cell, read back for output checks. inch()
# is unusable here: on a wide build it returns the low 8 bits of the
Expand Down Expand Up @@ -435,7 +449,7 @@ def test_in_wstr(self):
'na\u00efve \u00a4', # ISO-8859-1
'soup\u00e7on \u20ac', # ISO-8859-15
'\u0434\u044f\u043a']: # KOI8-U
if self._encodable(s):
if self._storable(s):
with self.subTest(s=s):
stdscr.addstr(0, 0, s)
self.assertEqual(stdscr.in_wstr(0, 0, len(s)), s)
Expand All @@ -450,7 +464,7 @@ def test_complexchar(self):
self.assertTrue(cc.attr & curses.A_BOLD)
self.assertEqual(cc.pair, 0)
# A spacing character optionally followed by combining characters.
if self._encodable('e\u0301'):
if self._storable('e\u0301'):
self.assertEqual(str(curses.complexchar('e\u0301')), 'e\u0301')
# Defaults: no attributes, color pair 0.
cc = curses.complexchar('z')
Expand Down Expand Up @@ -496,7 +510,7 @@ def test_in_wch(self):
self.assertTrue(cc.attr & curses.A_UNDERLINE)
# A character round-trips through the cell. See _encodable for the set.
for ch in ('A', '\u00e9', '\u00a4', '\u20ac', '\u0454'):
if self._encodable(ch):
if self._storable(ch):
with self.subTest(ch=ch):
stdscr.addch(3, 0, curses.complexchar(ch))
self.assertEqual(str(stdscr.in_wch(3, 0)), ch)
Expand Down Expand Up @@ -530,7 +544,7 @@ def test_getbkgrnd(self):
self.assertTrue(cc.attr & curses.A_BOLD)
# A non-ASCII background round-trips as a complexchar. See _encodable.
for ch in ('é', '¤', '€', 'є'):
if self._encodable(ch):
if self._storable(ch):
with self.subTest(ch=ch):
stdscr.bkgd(curses.complexchar(ch))
self.assertEqual(str(stdscr.getbkgrnd()), ch)
Expand Down Expand Up @@ -569,7 +583,7 @@ def test_complexstr(self):
self.assertNotEqual(s, curses.complexstr([cc('A'), 'b', cc('c')]))
self.assertNotEqual(s, curses.complexstr([cc('A', B), 'b']))
# A spacing character optionally followed by combining characters.
if self._encodable('é'):
if self._storable('é'):
self.assertEqual(str(curses.complexstr(['é', 'x'])),
'éx')
# cells is positional-only.
Expand All @@ -586,7 +600,9 @@ def test_complexstr(self):
self.assertEqual(str(curses.complexstr('abc')), 'abc')
self.assertEqual(len(curses.complexstr('')), 0)
base = 'é' # 'e' + combining acute: two code points, one cell
if self._encodable(base):
# Combining sequences need wide-character cells (a narrow build stores
# one byte per cell).
if hasattr(curses.window, 'get_wch') and self._encodable(base):
self.assertEqual(len(curses.complexstr(base)), 1)
self.assertEqual(curses.complexstr(base)[0], cc(base))
self.assertEqual(len(curses.complexstr('a' + base + 'b')), 3)
Expand Down Expand Up @@ -734,7 +750,7 @@ def test_output_character(self):
# str is stored as a wide-character cell on a wide build, so every
# encodable character round-trips, insch() included. A multibyte
# character does not fit a cell on a narrow build and is skipped.
wide = hasattr(stdscr, 'in_wch')
wide = hasattr(stdscr, 'get_wch')
for c in ('é', '¤', '€', 'є'):
if not self._encodable(c):
continue
Expand Down Expand Up @@ -2219,6 +2235,68 @@ def test_textbox_fill_last_cell_scrollok(self):
self._type(box, 'def')
self.assertEqual(box.gather(), 'abc\ndef\n')

def test_textbox_8bit(self):
# A character of an 8-bit locale encoding is entered and read back
# through the byte API. The byte path also runs on a wide build, so the
# test is not skipped there. Run the suite under an 8-bit locale
# (ISO-8859-1, ISO-8859-15 or KOI8-U) to reach the non-ASCII cases; each
# string is used only if the encoding maps it to single bytes. 'abc' is
# ASCII, 'café' is common to the Latin encodings, and the rest are
# distinctive (byte 0xA4 is '¤'/'€'/'є' in ISO-8859-1/-15/KOI8-U).
encoding = self.stdscr.encoding
for text in ['abc', 'café', 'naïve ¤¦', 'café €Šž', 'дякую єі']:
try:
data = text.encode(encoding)
except UnicodeEncodeError:
continue
if len(data) != len(text):
continue # a multibyte encoding is not the 8-bit byte path
with self.subTest(text=text):
box, win = self._make_textbox(1, 16)
for byte in data:
box.do_command(byte)
self.assertEqual(box.gather(), text + ' ')

def test_textbox_8bit_insert(self):
# Insert mode shifts the rest of the line right by reading each cell back
# and rewriting it; a non-ASCII 8-bit-locale character must survive the
# shift, even on a wide build where inch() mangles it. See
# test_textbox_8bit for the character choices.
encoding = self.stdscr.encoding
for ch in ['é', '¤', '€', 'є']:
try:
data = ch.encode(encoding)
except UnicodeEncodeError:
continue
if len(data) != 1:
continue
with self.subTest(ch=ch):
box, win = self._make_textbox(1, 10, insert_mode=True)
for byte in ('a' + ch + 'c').encode(encoding):
box.do_command(byte)
win.move(0, 1)
box.do_command(ord('b')) # insert 'b', shifting ch and 'c' right
self.assertEqual(box.gather(), 'ab' + ch + 'c ')

def test_textbox_8bit_fill_last_cell(self):
# A non-ASCII 8-bit-locale character must survive being written to the
# lower-right cell, which uses insch() rather than addch(). See
# test_textbox_8bit for the character choices.
encoding = self.stdscr.encoding
for ch in ['é', '¤', '€', 'є']:
try:
data = ch.encode(encoding)
except UnicodeEncodeError:
continue
if len(data) != 1:
continue
with self.subTest(ch=ch):
text = 'ab' + ch # the last character fills the corner
box, win = self._make_textbox(1, len(text), stripspaces=0)
for byte in text.encode(encoding):
box.do_command(byte)
self.assertEqual(box.gather(), text)

def test_textbox_movement(self):
box, win = self._make_textbox(3, 10)
self._type(box, 'abc')
Expand Down Expand Up @@ -2596,6 +2674,11 @@ def setUp(self):
self.mock_win = MagicMock(spec=curses.window)
self.mock_win.getyx.return_value = (1, 1)
self.mock_win.getmaxyx.return_value = (10, 20)
self.mock_win.encoding = 'utf-8'
# A non-blank cell so that _end_of_line() reports a full line: instr()
# backs the text reads, inch() the insert-mode shift.
self.mock_win.instr.return_value = b'x'
self.mock_win.inch.return_value = ord('x')
self.textbox = curses.textpad.Textbox(self.mock_win)

def test_init(self):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,9 @@ cell -- a spacing character optionally followed by combining characters -- in
addition to a single integer or byte character. Add the wide-character read
methods :meth:`curses.window.get_wstr` and :meth:`curses.window.in_wstr`, and
the functions :func:`curses.erasewchar`, :func:`curses.killwchar` and
:func:`curses.wunctrl`. These features are only available when built against
the wide-character ncursesw library.
:func:`curses.wunctrl`. On a narrow (non-ncursesw) build the character cell
holds a single character without combining marks, representable as one byte in
the window's encoding, and :meth:`curses.window.in_wstr` returns its decoded
text; :meth:`curses.window.get_wstr` and the :func:`curses.erasewchar`,
:func:`curses.killwchar` and :func:`curses.wunctrl` functions require the
wide-character ncursesw library.
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
Add the :class:`curses.complexchar` type, representing a styled wide-character
Add the :class:`curses.complexchar` type, representing a styled character
cell (text, attributes and color pair), and the :mod:`curses` window methods
:meth:`~curses.window.in_wch` and :meth:`~curses.window.getbkgrnd` that return
one. The character-cell methods (:meth:`~curses.window.addch`,
:meth:`~curses.window.bkgd`, :meth:`~curses.window.border`,
:meth:`~curses.window.hline` and others) now also accept a
:class:`~curses.complexchar`.
:class:`~curses.complexchar`. This works whether or not Python was built
against a wide-character-aware curses library; on a narrow build a cell holds a
single character representable as one byte in the window's encoding.
Loading
Loading