py-tokenize-rt-6.2.0

round-tripping wrapper for Python stdlib's "tokenize"

Index | devel

Description

The stdlib tokenize module does not properly roundtrip. This wrapper
around the stdlib provides two additional tokens ESCAPED_NL and
UNIMPORTANT_WS, and a Token data type. Use src_to_tokens and
tokens_to_src to roundtrip.

This library is useful if you're writing a refactoring tool based
on the Python tokenization.

Homepage: https://github.com/asottile/tokenize-rt