What’s New In Python 3.6
Release: | 3.6.0a3 |
---|---|
Date: | July 28, 2016 |
В данной статье описываются новые функции в Python 3.6, по сравнению с 3.5.
For full details, see the Misc/NEWS file.
Note
Prerelease users should be aware that this document is currently in draft form. It will be updated substantially as Python 3.6 moves towards release, so it’s worth checking back even after reading earlier versions.
Summary – Release highlights
New syntax features:
- PEP 498: Formatted string literals
Windows improvements:
- The
py.exe
launcher, when used interactively, no longer prefers Python 2 over Python 3 when the user doesn’t specify a version (via command line arguments or a config file). Handling of shebang lines remains unchanged - “python” refers to Python 2 in that case.
New Features
PEP 498: Formatted string literals
Formatted string literals are a new kind of string literal, prefixed
with ‘f’
. They are similar to the format strings accepted by
str.format(). They contain replacement fields surrounded by
curly braces. The replacement fields are expressions, which are
evaluated at run time, and then formatted using the format() protocol.
>>> name = "Fred" >>> f"He said his name is {name}." 'He said his name is Fred.'
See PEP 498 and the main documentation at Formatted string literals.
PYTHONMALLOC environment variable
The new PYTHONMALLOC environment variable allows setting the Python memory allocators and/or install debug hooks.
It is now possible to install debug hooks on Python memory allocators on Python
compiled in release mode using PYTHONMALLOC=debug
. Effects of debug hooks:
- Newly allocated memory is filled with the byte
0xCB
- Freed memory is filled with the byte
0xDB
- Detect violations of Python memory allocator API. For example,
PyObject_Free()
called on a memory block allocated by PyMem_Malloc(). - Detect write before the start of the buffer (buffer underflow)
- Detect write after the end of the buffer (buffer overflow)
- Check that the GIL is held when allocator
functions of PYMEM_DOMAIN_OBJ (ex:
PyObject_Malloc()
) and PYMEM_DOMAIN_MEM (ex: PyMem_Malloc()) domains are called.
Checking if the GIL is held is also a new feature of Python 3.6.
See the PyMem_SetupDebugHooks() function for debug hooks on Python memory allocators.
It is now also possible to force the usage of the malloc()
allocator of
the C library for all Python memory allocations using PYTHONMALLOC=malloc
.
It helps to use external memory debuggers like Valgrind on a Python compiled in
release mode.
On error, the debug hooks on Python memory allocators now use the tracemalloc module to get the traceback where a memory block was allocated.
Example of fatal error on buffer overflow using
python3.6 -X tracemalloc=5
(store 5 frames in traces):
Debug memory block at address p=0x7fbcd41666f8: API 'o' 4 bytes originally requested The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected. The 8 pad bytes at tail=0x7fbcd41666fc are not all FORBIDDENBYTE (0xfb): at tail+0: 0x02 *** OUCH at tail+1: 0xfb at tail+2: 0xfb at tail+3: 0xfb at tail+4: 0xfb at tail+5: 0xfb at tail+6: 0xfb at tail+7: 0xfb The block was made by call #1233329 to debug malloc/realloc. Data at p: 1a 2b 30 00 Memory block allocated at (most recent call first): File "test/test_bytes.py", line 323 File "unittest/case.py", line 600 File "unittest/case.py", line 648 File "unittest/suite.py", line 122 File "unittest/suite.py", line 84 Fatal Python error: bad trailing pad byte Current thread 0x00007fbcdbd32700 (most recent call first): File "test/test_bytes.py", line 323 in test_hex File "unittest/case.py", line 600 in run File "unittest/case.py", line 648 in __call__ File "unittest/suite.py", line 122 in run File "unittest/suite.py", line 84 in __call__ File "unittest/suite.py", line 122 in run File "unittest/suite.py", line 84 in __call__ ...
(Contributed by Victor Stinner in issue 26516 and issue 26564.)
Other Language Changes
- None yet.
New Modules
- None yet.
Improved Modules
asyncio
Since the asyncio module is provisional, all changes introduced in Python 3.6 have also been backported to Python 3.5.x.
Notable changes in the asyncio module since Python 3.5.0:
- The ensure_future() function and all functions that use it, such as loop.run_until_complete(), now accept all kinds of awaitable objects. (Contributed by Yury Selivanov.)
- New run_coroutine_threadsafe() function to submit coroutines to event loops from other threads. (Contributed by Vincent Michel.)
- New Transport.is_closing() method to check if the transport is closing or closed. (Contributed by Yury Selivanov.)
- The loop.create_server() method can now accept a list of hosts. (Contributed by Yann Sionneau.)
- New loop.create_future() method to create Future objects. This allows alternative event loop implementations, such as uvloop, to provide a faster asyncio.Future implementation. (Contributed by Yury Selivanov.)
- New loop.get_exception_handler() method to get the current exception handler. (Contributed by Yury Selivanov.)
- New
timeout()
context manager to simplify timeouts handling code. (Contributed by Andrew Svetlov.) - New StreamReader.readuntil() method to read data from the stream until a separator bytes sequence appears. (Contributed by Mark Korenberg.)
- The loop.getaddrinfo()
method is optimized to avoid calling the system
getaddrinfo
function if the address is already resolved. (Contributed by A. Jesse Jiryu Davis.)
contextlib
The contextlib.AbstractContextManager class has been added to
provide an abstract base class for context managers. It provides a
sensible default implementation for __enter__() which returns
self
and leaves __exit__() an abstract method. A matching
class has been added to the typing module as
typing.ContextManager.
(Contributed by Brett Cannon in issue 25609.)
datetime
The datetime.strftime() and
date.strftime() methods now support ISO 8601 date
directives %G
, %u
and %V
.
(Contributed by Ashley Anderson in issue 12006.)
faulthandler
On Windows, the faulthandler module now installs a handler for Windows exceptions: see faulthandler.enable(). (Contributed by Victor Stinner in issue 23848.)
idlelib and IDLE
The idlelib package is being modernized and refactored to make IDLE look and work better and to make the code easier to understand, test, and improve. Part of making IDLE look better, especially on Linux and Mac, is using ttk widgets, mostly in the dialogs. As a result, IDLE no longer runs with tcl/tk 8.4. It now requires tcl/tk 8.5 or 8.6. We recommend running the latest release of either.
'Modernizing' includes renaming and consolidation of idlelib modules. The renaming of files with partial uppercase names is similar to the renaming of, for instance, Tkinter and TkFont to tkinter and tkinter.font in 3.0. As a result, imports of idlelib files that worked in 3.5 will usually not work in 3.6. At least a module name change will be needed (see idlelib/README.txt), sometimes more. (Name changes contributed by Al Swiegart and Terry Reedy in issue 24225. Most idlelib patches since have been and will be part of the process.)
In compensation, the eventual result with be that some idlelib classes will be easier to use, with better APIs and docstrings explaining them. Additional useful information will be added to idlelib when available.
importlib
importlib.util.LazyLoader now calls create_module() on the wrapped loader, removing the restriction that importlib.machinery.BuiltinImporter and importlib.machinery.ExtensionFileLoader couldn't be used with importlib.util.LazyLoader.
os
A new close() method allows explicitly closing a
scandir() iterator. The scandir() iterator now
supports the context manager protocol. If a scandir()
iterator is neither exhausted nor explicitly closed a ResourceWarning
will be emitted in its destructor.
(Contributed by Serhiy Storchaka in issue 25994.)
pickle
Objects that need calling __new__
with keyword arguments can now be pickled
using pickle protocols older than protocol version 4.
Protocol version 4 already supports this case. (Contributed by Serhiy
Storchaka in issue 24164.)
readline
Added set_auto_history() to enable or disable automatic addition of input to the history list. (Contributed by Tyler Crompton in issue 26870.)
rlcompleter
Private and special attribute names now are omitted unless the prefix starts with underscores. A space or a colon is added after some completed keywords. (Contributed by Serhiy Storchaka in issue 25011 and issue 25209.)
Names of most attributes listed by dir() are now completed. Previously, names of properties and slots which were not yet created on an instance were excluded. (Contributed by Martin Panter in issue 25590.)
site
When specifying paths to add to sys.path in a .pth file, you may now specify file paths on top of directories (e.g. zip files). (Contributed by Wolfgang Langner in issue 26587).
sqlite3
- sqlite3.Cursor.lastrowid now supports the
REPLACE
statement. (Contributed by Alex LordThorsen in issue 16864.)
socket
The ioctl() function now supports the SIO_LOOPBACK_FAST_PATH control code. (Contributed by Daniel Stokes in issue 26536.)
socketserver
Servers based on the socketserver module, including those defined in http.server, xmlrpc.server and wsgiref.simple_server, now support the context manager protocol. (Contributed by Aviv Palivoda in issue 26404.)
The wfile
attribute of
StreamRequestHandler classes now implements
the io.BufferedIOBase writable interface. In particular,
calling write() is now guaranteed to send the
data in full. (Contributed by Martin Panter in issue 26721.)
subprocess
subprocess.Popen destructor now emits a ResourceWarning warning
if the child process is still running. Use the context manager protocol (with
proc: ...
) or call explicitly the wait() method to
read the exit status of the child process (Contributed by Victor Stinner in
issue 26741).
telnetlib
Telnet is now a context manager (contributed by Stéphane Wirtel in issue 25485).
tkinter
Added methods trace_add()
,
trace_remove()
and trace_info()
in the tkinter.Variable
class. They replace old methods
trace_variable()
, trace()
,
trace_vdelete()
and
trace_vinfo()
that use obsolete Tcl commands and might
not work in future versions of Tcl.
(Contributed by Serhiy Storchaka in issue 22115).
typing
The typing.ContextManager class has been added for representing contextlib.AbstractContextManager. (Contributed by Brett Cannon in issue 25609.)
unittest.mock
The Mock class has the following improvements:
- Two new methods, Mock.assert_called() and Mock.assert_called_once() to check if the mock object was called. (Contributed by Amit Saha in issue 26323.)
urllib.robotparser
RobotFileParser now supports the Crawl-delay
and
Request-rate
extensions.
(Contributed by Nikolay Bogoychev in issue 16099.)
warnings
A new optional source parameter has been added to the
warnings.warn_explicit() function: the destroyed object which emitted a
ResourceWarning. A source attribute has also been added to
warnings.WarningMessage
(contributed by Victor Stinner in
issue 26568 and issue 26567).
When a ResourceWarning warning is logged, the tracemalloc is now used to try to retrieve the traceback where the detroyed object was allocated.
Example with the script example.py
:
import warnings def func(): return open(__file__) f = func() f = None
Output of the command python3.6 -Wd -X tracemalloc=5 example.py
:
example.py:7: ResourceWarning: unclosed file <_io.TextIOWrapper name='example.py' mode='r' encoding='UTF-8'> f = None Object allocated at (most recent call first): File "example.py", lineno 4 return open(__file__) File "example.py", lineno 6 f = func()
The "Object allocated at" traceback is new and only displayed if tracemalloc is tracing Python memory allocations and if the warnings was already imported.
winreg
Added the 64-bit integer type REG_QWORD. (Contributed by Clement Rouault in issue 23026.)
zipfile
A new ZipInfo.from_file() class method allows making a ZipInfo instance from a filesystem file. A new ZipInfo.is_dir() method can be used to check if the ZipInfo instance represents a directory. (Contributed by Thomas Kluyver in issue 26039.)
The ZipFile.open() method can now be used to write data into a ZIP file, as well as for extracting data. (Contributed by Thomas Kluyver in issue 26039.)
zlib
The compress() function now accepts keyword arguments. (Contributed by Aviv Palivoda in issue 26243.)
fileinput
hook_encoded() now supports the errors argument. (Contributed by Joseph Hackman in issue 25788.)
Optimizations
- The ASCII decoder is now up to 60 times as fast for error handlers
surrogateescape
,ignore
andreplace
(Contributed by Victor Stinner in issue 24870). - The ASCII and the Latin1 encoders are now up to 3 times as fast for the
error handler
surrogateescape
(Contributed by Victor Stinner in issue 25227). - The UTF-8 encoder is now up to 75 times as fast for error handlers
ignore
,replace
,surrogateescape
,surrogatepass
(Contributed by Victor Stinner in issue 25267). - The UTF-8 decoder is now up to 15 times as fast for error handlers
ignore
,replace
andsurrogateescape
(Contributed by Victor Stinner in issue 25301). bytes % args
is now up to 2 times faster. (Contributed by Victor Stinner in issue 25349).bytearray % args
is now between 2.5 and 5 times faster. (Contributed by Victor Stinner in issue 25399).- Optimize bytes.fromhex() and bytearray.fromhex(): they are now between 2x and 3.5x faster. (Contributed by Victor Stinner in issue 25401).
- Optimize
bytes.replace(b'', b'.')
andbytearray.replace(b'', b'.')
: up to 80% faster. (Contributed by Josh Snider in issue 26574). - Allocator functions of the PyMem_Malloc() domain
(PYMEM_DOMAIN_MEM) now use the pymalloc memory allocator instead of
malloc()
function of the C library. The pymalloc allocator is optimized for objects smaller or equal to 512 bytes with a short lifetime, and usemalloc()
for larger memory blocks. (Contributed by Victor Stinner in issue 26249). - pickle.load() and pickle.loads() are now up to 10% faster when deserializing many small objects (Contributed by Victor Stinner in issue 27056).
Build and C API Changes
- New Py_FinalizeEx() API which indicates if flushing buffered data failed (issue 5319).
- PyArg_ParseTupleAndKeywords() now supports positional-only parameters. Positional-only parameters are defined by empty names. (Contributed by Serhiy Storchaka in issue 26282).
Deprecated
New Keywords
async
and await
are not recommended to be used as variable, class,
function or module names. Introduced by PEP 492 in Python 3.5, they will
become proper keywords in Python 3.7.
Deprecated Python modules, functions and methods
- importlib.machinery.SourceFileLoader.load_module() and importlib.machinery.SourcelessFileLoader.load_module() are now deprecated. They were the only remaining implementations of importlib.abc.Loader.load_module() in importlib that had not been deprecated in previous versions of Python in favour of importlib.abc.Loader.exec_module().
Deprecated functions and types of the C API
- None yet.
Deprecated features
- The
pyvenv
script has been deprecated in favour ofpython3 -m venv
. This prevents confusion as to what Python interpreterpyvenv
is connected to and thus what Python interpreter will be used by the virtual environment. (Contributed by Brett Cannon in issue 25154.) - When performing a relative import, falling back on
__name__
and__path__
from the calling module when__spec__
or__package__
are not defined now raises an ImportWarning. (Contributed by Rose Ames in issue 25791.) - Unlike to other dbm implementations, the dbm.dumb module
creates database in
'r'
and'w'
modes if it doesn't exist and allows modifying database in'r'
mode. This behavior is now deprecated and will be removed in 3.8. (Contributed by Serhiy Storchaka in issue 21708.)
Deprecated Python behavior
- Raising the StopIteration exception inside a generator will now generate a DeprecationWarning, and will trigger a RuntimeError in Python 3.7. See PEP 479: Change StopIteration handling inside generators for details.
Removed
API and Feature Removals
inspect.getmoduleinfo()
was removed (was deprecated since CPython 3.3). inspect.getmodulename() should be used for obtaining the module name for a given path.traceback.Ignore
class andtraceback.usage
,traceback.modname
,traceback.fullmodname
,traceback.find_lines_from_code
,traceback.find_lines
,traceback.find_strings
,traceback.find_executable_lines
methods were removed from the traceback module. They were undocumented methods deprecated since Python 3.2 and equivalent functionality is available from private methods.- The
tk_menuBar()
andtk_bindForTraversal()
dummy methods in tkinter widget classes were removed (corresponding Tk commands were obsolete since Tk 4.0). - The open() method of the zipfile.ZipFile
class no longer supports the
'U'
mode (was deprecated since Python 3.4). Use io.TextIOWrapper for reading compressed text files in universal newlines mode.
Porting to Python 3.6
This section lists previously described changes and other bugfixes that may require changes to your code.
Changes in 'python' Command Behavior
- The output of a special Python build with defined
COUNT_ALLOCS
,SHOW_ALLOC_COUNT
orSHOW_TRACK_COUNT
macros is now off by default. It can be re-enabled using the-X showalloccount
option. It now outputs tostderr
instead ofstdout
. (Contributed by Serhiy Storchaka in issue 23034.)
Changes in the Python API
- When importlib.abc.Loader.exec_module() is defined, importlib.abc.Loader.create_module() must also be defined.
- The format of the
co_lnotab
attribute of code objects changed to support negative line number delta. By default, Python does not emit bytecode with negative line number delta. Functions usingframe.f_lineno
,PyFrame_GetLineNumber()
orPyCode_Addr2Line()
are not affected. Functions decoding directlyco_lnotab
should be updated to use a signed 8-bit integer type for the line number delta, but it's only required to support applications using negative line number delta. SeeObjects/lnotab_notes.txt
for theco_lnotab
format and how to decode it, and see the PEP 511 for the rationale. - The functions in the compileall module now return booleans instead
of
1
or0
to represent success or failure, respectively. Thanks to booleans being a subclass of integers, this should only be an issue if you were doing identity checks for1
or0
. See issue 25768. - Reading the
port
attribute of urllib.parse.urlsplit() and urlparse() results now raises ValueError for out-of-range values, rather than returning None. See issue 20059. - The imp module now raises a DeprecationWarning instead of PendingDeprecationWarning.
- The following modules have had missing APIs added to their
__all__
attributes to match the documented APIs: calendar, cgi, csv, ElementTree, enum, fileinput, ftplib, logging, mailbox, mimetypes, optparse, plistlib, smtpd, subprocess, tarfile, threading and wave. This means they will export new symbols whenimport *
is used. See issue 23883. - When performing a relative import, if
__package__
does not compare equal to__spec__.parent
then ImportWarning is raised. (Contributed by Brett Cannon in issue 25791.) - When a relative import is performed and no parent package is known, then ImportError will be raised. Previously, SystemError could be raised. (Contributed by Brett Cannon in issue 18018.)
- Servers based on the socketserver module, including those defined in http.server, xmlrpc.server and wsgiref.simple_server, now only catch exceptions derived from Exception. Therefore if a request handler raises an exception like SystemExit or KeyboardInterrupt, handle_error() is no longer called, and the exception will stop a single-threaded server. (Contributed by Martin Panter in issue 23430.)
- spwd.getspnam() now raises a PermissionError instead of KeyError if the user doesn't have privileges.
- The socket.socket.close() method now raises an exception if an error (e.g. EBADF) was reported by the underlying system call. See issue 26685.
- The decode_data argument for smtpd.SMTPChannel and
smtpd.SMTPServer constructors is now
False
by default. This means that the argument passed to process_message() is now a bytes object by default, andprocess_message()
will be passed keyword arguments. Code that has already been updated in accordance with the deprecation warning generated by 3.5 will not be affected. - All optional parameters of the dump(), dumps(), load() and loads() functions and JSONEncoder and JSONDecoder class constructors in the json module are now keyword-only. (Contributed by Serhiy Storchaka in issue 18726.)
Changes in the C API
- PyMem_Malloc() allocator family now uses the pymalloc allocator rather than system
malloc()
. Applications calling PyMem_Malloc() without holding the GIL can now crash. Set the PYTHONMALLOC environment variable todebug
to validate the usage of memory allocators in your application. See issue 26249. - Py_Exit() (and the main interpreter) now override the exit status with 120 if flushing buffered data failed. See issue 5319.