Python Language: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(645 intermediate revisions by the same user not shown)
Line 1: Line 1:
=External=
* https://docs.python.org/3/glossary.html
=Internal=
=Internal=
* [[Python#Subjects|Python]]
* [[Python#Subjects|Python]]
* [[Python Code Examples]]
* [[Python_Language_String#Overview|Strings]] | [[Python_Language_List#Overview|List]] | [[Python_Language_Dictionary#Overview|Dictionary]]
* [[Python_Language_Functions#Overview|Python Language | Functions]]
* [[Python_Language_Modularization#Overview|Python Language | Modularization]]
* [[Python_Language_Exceptions#Overview|Python Language | Exceptions]]
* [[Python_Language_OOP#Overview|Python Language | Object Oriented Programming]]
=TODO=
<font color=darkkhaki>
Expand the following sections and subjects:
* [[IPy]]
** [[Python_Language#Iterate_Multiple_Sequences_with_zip.28.29]]
** [[Python_Language#Generate_Number_Sequences_with_range.28.29]]
** [[Python_Language#Iterate_over_Code_Structures_with_itertools]]
** [[Printing_to_stdout_in_Python#pprint.28.29]]
</font>
=Overview=
Python is a general-purpose, high-level, dynamically-typed language. Its design makes it very readable. Its relative terseness makes it possible to write a program that is much smaller than the equivalent static language program. However, if the program is CPU-bound, a program written in C, C++ or Java will generally run faster than its Python equivalent. While Python offers thread abstractions via libraries, only one thread can hold the control of the Python interpreter, because of the [[Python Global Interpreter Lock (GIL)|Global Interpreter Lock (GIL)]].
Python installation contains the core language support (built-ins) and [[#Python_Standard_Library|standard libraries]].
Python programs can be executed in two modes: interactively via an interpreter, also called a shell, or stored into a file with the usual, but optional <code>.py</code> extension and run by typing <code>python</code> followed by the file name.
In Python, everything is an [[Python_Language_OOP#Object|object]]. This includes numbers, strings, tuples, [[Python_Language_List#Overview|lists]], dictionaries, [[Python_Language_Functions#Overview|functions]] and programs. The ID of any object can be obtained with the [[Python_Language_Functions#id.28.29|built-in function <code>id()</code>]]. The definition of an object is called [[Python_Language_OOP#Class|class]].
In Python, spacing does matter, [[#Sequential_Steps|sequential blocks]] are indented at the same level, and they are known as <span id='indented_block'></span>'''indented blocks'''. Python uses white space to define program structure.
=Python Version=
See: {{Internal|Python Versions#Overview|Python Versions}}
=Printing=
Printing done with the <code>print()</code> function in Python 3 (it uses to be a statement in Python 2).
<syntaxhighlight lang='py'>
print('something')
</syntaxhighlight>
Also see: {{Internal|Printing to stdout in Python#Overview|Printing to stdout in Python}}
=Comments=
<syntaxhighlight lang='py'>
# This is a comment
s = 0 # This is also a comment
</syntaxhighlight>
Everything from <code>#</code> to the end of the line is comment. Python does not have a multi-line comment.
=Code Line Continuation=
Python encourages short lines. Recommended maximum line length is 80 characters. To continue a line on the next line, use backslash:
<syntaxhighlight lang='py'>
s = 'A' + \
'B' + \
'C'
</syntaxhighlight>
Line continuation is needed if a Python expression spans multiple lines.
=Reserved Words=
=Reserved Words=
Reserved words can only be used to mean the thing Python expects them to mean. They cannot be used as [[#Variables|variable names]], [[#Functions|function names]], [[#Classes|class names]],  or [[#Identifiers|identifiers]].
Reserved words, or keywords, can only be used to mean the thing Python expects them to mean. They cannot be used as [[#Variables|variable names]], [[Python_Language_Functions#Function_Name_Rules|function names]], [[#Classes|class names]],  or [[#Identifiers|identifiers]].
{|
:::{|class="wikitable" style="text-align: left;"
| <code>False</code> || <code>class</code> || <code>return</code> || <code>is</code> || <code>finally</code>
| <font face='menlo' size='-2'>[[Python_Language_Exceptions#try.2Fexcept|try]]</font> || <font face='menlo' size='-2'>[[Python_Language_Exceptions#try.2Fexcept|except]]</font>|| <font face='menlo' size='-2'>[[Python_Language_Exceptions#finally|finally]]</font> || <font face='menlo' size='-2'>[[Python_Language_Exceptions#raiset|raise]]</font> || <font face='menlo' size='-2'>[[Python_Language_Exceptions#assertt|assert]]</font>
|-
|-
| <code>None</code> || <code>if</code> || <code>for</code> || <code>lambda</code> || <code>continue</code>
| <font face='menlo' size='-2'>[[#while|while]]</font> || <font face='menlo' size='-2'>[[#for|for]]</font> || <font face='menlo' size='-2'>[[#break|break]]</font> || <font face='menlo' size='-2'>[[#continue|continue]]</font> || <font face='menlo' size='-2'>[[#for|in]]</font>
|-
|-
| <code>True</code> || <code>def</code> || <code>from</code> || <code>while</code> || <code>nonlocal</code>
| <font face='menlo' size='-2'>[[#Constants|False]]</font> || <font face='menlo' size='-2'>[[#Constants|True]]</font> || <font face='menlo' size='-2'>[[#None|None]]</font> || ||  
|-
|-
| <code>and</code> || <code>del</code> || <code>global</code> || <code>not</code> || <code>with</code>
| <font face='menlo' size='-2'>[[#Conditional_Steps|if]]</font> || <font face='menlo' size='-2'>[[#Conditional_Steps|else]]</font> || <font face='menlo' size='-2'>[[#Conditional_Steps|elif]]</font> || ||
|-
|-
| <code>as</code> || <code>elif</code> || <code>try</code> || <code>or</code> || <code>yield</code>
| <span id='def'></span><font face='menlo' size='-2'>[[Python_Language_Functions#Function_Definition|def]]</font> || <font face='menlo' size='-2'>[[Python_Language_Functions#Return_Value|return]]</font> ||   <font face='menlo' size='-2'>[[Python_Language_Functions#Lambdas|lambda]]</font> || ||
|-
|-
| <code>assert</code> || <code>else</code> || <code>import</code> || <code>pass</code>  
| <font face='menlo' size='-2'>[[#and|and]]</font> || <font face='menlo' size='-2'>[[#or|or]]</font> || <font face='menlo' size='-2'>[[#not|not]]</font>  || ||
|-
| <font face='menlo' size='-2'>[[Python_Language_Modularization#Importing|import]]</font> || <font face='menlo' size='-2'>[[Python_Language_Modularization#Importing|from]]</font> || || ||
|-
|  <span id='with'></span><code>[[Python_Context_Manager#How_it_Works|with]]</code> || <font face='menlo' size='-2'>[[#as|as]]</font> || ||  ||
|-
| <font face='menlo' size='-2'>[[#nonlocal|nonlocal]]</font> || <font face='menlo' size='-2'><span id='yield'></span>[[Python_Generators#Overview|yield]]</font> ||  || ||
|-
| <font face='menlo' size='-2'>[[#pass|pass]]</font> || || || ||
|-
| <font face='menlo' size='-2'>[[#is|is]]</font> || || || ||
|-
|  <span id='del'></span><font face='menlo' size='-2'>[[Python_Language_Dictionary#Delete_Elements|del]]</font> ||  || || ||
|-
| <span id='class'></span><font face='menlo' size='-2'>[[Python_Language_OOP#Class|class]]</font>|| || || ||
|-
| <span id='global'></span><font face='menlo' size='-2'>[[#Global_Usage|global]]</font>|| || || ||
|-
|-
| <code>break</code> || <code>except</code> || <code>in</code> || <code>raise</code>
|}
|}
Also see: {{Internal|Variables,_Parameters,_Arguments#Keyword_.28Reserved_Word.29|Keywords}}
====<span id='as'></span>The <tt>as</tt> Reserved Word====
Used:
* in the [[Python_Language_Modularization#Import_As|<code>import</code> statement]], to rename the module being imported
* With the <code>[[Python_Context_Manager#How_it_Works|with]]</code> reserved word, to designate an object returned by the [[Python_Context_Manager#Overview|context manager]].
* in exception handling blocks [[Python_Language_Exceptions#Except_As |to assign an exception to a variable]]
=Literals=
Literals have a [[#Type|type]].
==Integer Literals==
==String Literals==
{{Internal|Python_Language_String#Quotes|Strings &#124; Quotes}}
==F-String==
{{Internal|Python_Language_String#F-String|F-String}}
=<span id='Constant'></span>Constants=
Constants are fixed values, they do not change throughout the program. Constants can be boolean (<code>True</code>, <code>False</code>), numeric (integers or floating point numbers), or strings, which can be single quoted or double quoted, or even "the absence of a value" ([[#None|None]]). Constants can be assigned to [[#Variables|variables]], can be arguments of [[#Functions|functions]]. Constants have a [[#Type|type]].


=Constants=
==Constant Variable==
Constants are fixed values, they do not change throughout the program. Constants can be numeric (integers or floating point numbers), or strings, which can be single quoted or double quoted.
 
Python does not have anything equivalent to <code>final</code> in Java, so any variable can be modified after assignment. You can use:
<syntaxhighlight lang='python'>
CONST_NAME = "Name"
</syntaxhighlight>
for clarity, but nothing prevents <code>CONST_NAME</code> to be assigned other value later in the program.
 
==<span id='Quotes'></span>Single Quoted, Double Quoted, Triple Single Quoted, Triple Double Quoted==
<font color=darkkhaki>TODO</font>
 
==Multi-Line Strings==
<font color=darkkhaki>TODO, see [[Python Language Functions#Docstring]]</font>
===Triple Double-Quoted Strings===
Triple double-quoted strings is the recommended style for [[Python Language Functions#Docstring|docstrings]].
<syntaxhighlight lang='python'>
s = """
This is a
  multi-line
  string
"""
</syntaxhighlight>
 
=<span id='Variable'></span>Variables=
Variables are names associated with memory locations used to store values. Variables and [[#Identifier|identifiers]] are equivalent. Variables are declared and assigned a value through an [[#Assignment_Statement|assignment statement]]. Assignment is also referred to as '''binding''', as we are binding a name to an object. Variable names that have been assigned may be occasionally referred to as "bound variables".
 
The assignment does not copy a value, it just attaches the name to the object that contains the data. See [[#References|reference variables]], below.
 
<span id='Dynamically_Typed'></span>A variable may be associated with an object that has a [[#Type|type]], and the same variable may be assigned later an object of a different type. Python variables do not have an inherent type associated with them, a variable can refer to a different type of object simply by doing an assignment. The type information is stored in the object itself. A useful mental representation of a variable is a sticky note that can be attached to an object, and then re-attached to a different object, not necessarily of the same type.
<syntaxhighlight lang='py'>
a = 1
b = 'something'
print(a)
print(b)
a = 'something else'
print(a)
</syntaxhighlight>
 
Also see: {{Internal|Variables,_Parameters,_Arguments#Variable|Variables, Parameters, Arguments}}
==<span id='References'></span><span id='Reference'></span>All Python Variables are Reference Variables==
All Python variables are reference variables. When objects are passed as arguments to a function, new local variables are created on the function stack, referencing the original objects without any copying. For more details, see: {{Internal|Variables,_Parameters,_Arguments#Reference|Variables, Parameters, Arguments &#124; Reference}}


=Variables=
Variables are memory locations used to store values, and have labels associated to them , the '''variable name'''. Variables are declared and assigned a value though an [[#Assignment_Statement|assignment statement]].
==Variable Naming Rules==
==Variable Naming Rules==
Variable names are case sensitive. Variable names can start with letters or underscore ('_') - but underscores should be generally avoided because Python tends to use underscores for its internal purposes. The rest of the variable name can be letters, numbers and underscores. No other characters are allowed. Variable names should be sensible (mnemonic).
Variable names are case sensitive. Variable names can start with letters or underscore ('_') - but underscores should be generally avoided because Python treats names that begin with an underscore in special ways, and tends to use them for its internal purposes. The rest of the variable name can be lowercase letters (a through z), uppercase letters (A through Z), digits (0 through 9) and underscores. No other characters are allowed. Python has a set of words, called [[#Reserved_Words|reserved words]], that cannot be used as variable names. Variable names should be sensible (mnemonic). [[Python_Language_Functions#Function_Name_Rules|Function name]] follow the same rules.
 
Variables declared in functions should be lowercase. For more details see: {{Internal|Python Style|Python Style}}
 
===Leading Underscore Variable Names===
See:
* [[Python_Language_OOP_Attributes_and_Properties#.22Protected.22_Attributes|"Protected" class instance attributes]].
* "Protected" module variables and functions.
 
===Two Underscore (__) Variables===
Names that being and end with two underscores (__) are reserved for use within Python, for built-in variables.
====<tt>__name__</tt>====
<code>__name__</code> is a built-in variable which evaluates to the name of the current module and it is set differently depending on whether the module is imported or executed. For more details see: {{Internal|Python_Language_Modularization#Distinguishing_between_Importing_and_Executing_a_Module|Distinguishing between Importing and Executing a Module}}
====<tt>__package__</tt>====
{{External|https://docs.python.org/3/reference/import.html#package__}}
{{External|https://www.python.org/dev/peps/pep-0366}}
<code>__package__</code> is a module-level attribute, which must be set.  Its value must be a string, but it can be the same value as its <code>[[#name|__name__]]</code>. When the module is a package, its <code>__package__</code> value should be set to its <code>__name__</code>. When the module is not a package, <code>__package__</code> should be set to the empty string for top-level modules, or for submodules, to the parent package’s name. When it is present, relative imports will be based on this attribute rather than the module <code>[[#name|__name__]]</code> attribute.
 
====<tt>__file__</tt>====
Contains the name of the file the function was loaded from.
====<tt>__doc__</tt>====
====<tt>__loader__</tt>====
====<tt>__spec__</tt>====
====<tt>__annotations__</tt>====
====<tt>__builtins__</tt>====
 
==<span id='Variable_Scope'></span><span id='Namespace'></span>Variables Namespace and Scope==
A namespace is a collection of names that can be used to access various underlying Python objects, such as data instances, functions, classes, etc. Multiple namespaces can co-exist at a given time, but they are isolated from each other. Within a namespace, a particular name is unique. A Python program has three kinds of namespaces: [[#Built-in_Namespace|built-in]], [[#Global_Namespace|global]] and [[#Local_Namespace|local]].
 
::::[[File:Python_Namespaces.png]]
 
A module also has a [[Python_Language_Modularization#Global_Namespace|global namespace]].
 
A scope is the portion of a program where a namespace can be accessed directly without any prefix. At any given moment, there are at least three nested scopes: the scope fo the current function that has local names, the scope of the module, which has global names and the outermost scope that has built-in names. When a reference is made inside a function, the name is searched in the local namespace, the global namespace and finally in the built-in namespace. If there is a function inside another function, the embedded function's local scope is nested in the upper level function's local scope.
 
The variables declared in higher scopes can be read, without any prefix, but unless they are prefixed with the namespace they belong to, they cannot be modified. Only the variables in the same scope can be modified. If we try to modify a variable from a higher scope without prefixing it (which we can read without prefixing it), a local variable with the same name is created and that variable is written. For more details see [[#Interaction_between_Local_and_Global_Variables|Interaction between Local and Global Variables]], below.
 
===Built-in Namespace===
The built-in namespace contains all the built-in Python runtime names, such those of the [[Python_Language_Functions#Built-in_Functions|built-in functions]], for example. This namespace is create when the Python interpreter is started and exists for as long as the interpreter runs. The built-in namespace is the reason that built-in functions like <code>id()</code> and <code>print()</code> are always available in any part of the program, such a module, a function or a class.
===<span id='Global_Variable'></span><span id='Global_Variables'></span><span id='Global_Namespace'></span>Global Namespace and Variables===
Each module creates its own global namespace for its variables, functions and classes. Since the namespaces are isolated, the same name may exist in different modules and it does not collide. The variables defined in the global namespace, outside any function or class, are known as global variables. Global variables can be used by everyone within the module, both inside and outside of functions. The global variables are maintained by Python in the module's [[#Global_Symbol_Table|global symbol table]].
====<span id='Global_Symbol_Table'></span>Global Symbol Table and <tt>globals()</tt>====
The Python runtime maintains a global symbol table, which is a data structure that contains variable names and their association with objects in the global namespace of a module. The global symbol table contains all variables that are not associated with any class or function. The [[Python_Language_Functions#Built-in_Functions|built-in function]] <code>globals()</code> returns a dictionary of the contents of the global symbol table. Assuming that the main program contains a <code>some_func</code> function declaration and a <code>SomeClass</code> class declaration, the content of the global symbol table of the corresponding module is similar to:
<syntaxhighlight lang='text'>
__name__        = '__main__'
__doc__        = None
__package__    = None
__loader__      = <_frozen_importlib_external.SourceFileLoader object at 0x1090d5fa0>
__spec__        = None
__annotations__ = {}
__builtins__    = <module 'builtins' (built-in)>
__file__        = './main.py'
__cached__      = None
some_func      = <function some_func at 0x101dd4310>
SomeClass      = <class '__main__.SomeClass'>
</syntaxhighlight>
 
===<span id='Local_Variable'></span><span id='Local_Variables'></span><span id='Local_Namespace'></span>Local Namespaces and Variables===
Each [[Python_Language_Functions#Overview|function]] and class define their own namespace, which is created when the function is called or <font color=darkkhaki>the class is instantiated</font>. Variables defined inside a function are distinct from the variables with the same name defined in the main program, and they are known as local variables. However, the global variables are accessible within functions. Upon access from a function, the global variable with the same name will remain as it was, global and with the original value. Special rules apply when a local variable has the same name as an existing global variable. In this case we say that the local variable temporarily overrides the global variable, and this interaction is described in the [[#Interaction_between_Local_and_Global_Variables|Interaction between Local and Global Variables]] section.
====<span id='Local_Symbol_Table'></span>Local Symbol Table and <code>locals()</code>====
Each function <font color=darkkhaki>and class</font> contains a local symbol table, which maintains the names of the variables in the local namespace. The [[Python_Language_Functions#Built-in_Functions|built-in function]] <code>locals()</code> returns a dictionary of the contents of the local namespace.
 
===Get Names in a Namespace with <tt>dir()</tt>===
The [[Python_Language_Functions#Built-in_Functions|built-in function]] <code>dir()</code> returns a list of defined name in a namespace. Without arguments, it produces an alphabetically sorted list of names in the current local symbol table. When given an argument that is the name of an imported module, <code>dir()</code> lists the names defined in the imported module.
 
===Interaction between Local and Global Variables===
If a global variable is accessed inside a function, and then there is an attempt to change it, the program throws an <code>UnboundLocalError</code>:
<syntaxhighlight lang='py'>
a = 10
 
def some_func():
    print('a:', a)
    a = 11
 
some_func() # this throws UnboundLocalError
</syntaxhighlight>
<syntaxhighlight lang='text'>
Traceback (most recent call last):
  File "./main.py", line 8, in <module>
    some_func()
  File "./main.py", line 5, in some_func
    print('a:', a)
UnboundLocalError: local variable 'a' referenced before assignment
</syntaxhighlight>
If a variable with the same name as a global variable is modified without accessing the global variable first, this implicitly declares the local variable with the same name, and the program works fine:
<syntaxhighlight lang='py'>
a = 10
 
def some_func():
    a = 11
    print('a:', a)
 
some_func()
</syntaxhighlight>
This will display:
<syntaxhighlight lang='text'>
a: 11
</syntaxhighlight>
<span id='Global_Usage'></span>To explicitly indicate that we want to use the global variable inside the function, use the [[#global|<code>global</code> reserved word]]. This will tell the interpreter that we want to use the global variable, which then can be modified inside the function:
<syntaxhighlight lang='py'>
a = 10
 
def some_func():
    global a
    print("global 'a' from within the function:", a)
    a = 11
 
some_func()
print("global 'a' after function call:", a)
</syntaxhighlight>
If <code>global</code> is not explicitly declared inside a function, Python uses the local namespace and the variable is implicitly local. Local variables are created during the function execution and discarded after the function completes.
 
<code>global</code> keyword use is discouraged. Typically, global variables are used to store some kind of state in a system. If you find yourself using a lot of them, it may indicate a need for object-oriented programming.
===<tt>nonlocal</tt>===
<font color=darkkhaki>TODO <code>nonlocal</code>.</font>
 
=<span id='Identifier'></span>Identifiers=
An identifier is a name given to objects. Everything in Python is an object, so the name is the way to access the underlying object. An identifier and a [[#Variable|variable]] are equivalent.
 
=Type=
<span id='Python_is_Dynamically_Typed></span>Python is a [[Programming_Languages_Concepts#Dynamic_Typing|dynamically typed]] language, in that the same variable [[#Dynamically_Typed|may be bound]] to an object of a certain type, then to another object of a different type, at different moments in the execution of the program. Everything in Python is implemented as an object, and any object has a type, but the type information is stored in the object, not in the variable. The type determines whether the data value of the object is mutable or immutable (constant).
 
<span id='Python_is_Strongly_Typed></span>Python is [[Programming_Languages_Concepts#Strong_Typing|strongly typed]], which means that the type of an object does not change, even if the value is mutable. Strong typing is a property of the object instances, not of variables that are bound to those objects.
 
The type of an object can be obtained with the built-in function <code>[[Python_Language_Functions#type.28.29|type()]]</code> applied to the [[#Variable|variable]] or a [[#Constant|constant]] the object is assigned to. In Python, "[[Python_Language_OOP#Class|class]]" and "type" mean pretty much the same thing.
==<span id='Types'></span>Data Types==
Scalar types, or scalars, are "single value" types: <code>[[#Booleans|bool]]</code>, [[#Numbers|numbers]].
===None===
<syntaxhighlight lang='python'>
x = None
type(x)
<class 'NoneType'>
</syntaxhighlight>
<code>None</code> is a special Python value that holds a place when there is nothing to say. It is not the same as <code>False</code>, although it looks false when it is evaluated as a boolean. <code>None</code> can be used with the <code>[[#is|is]]</code> or <code>[[#is_not|is not]]</code> operators:
<syntaxhighlight lang='python'>
if x is None:
  ...
if x is not None:
  ...
</syntaxhighlight>
<code>None</code> is [[Python_Language_Functions#Return|returned]] by a function that does not contain the <code>return</code> statement. <code>None</code> is useful to distinguish a missing value from an empty value. Zero-value integers or floats, empty strings (&#39;'), empty lists <code>[]</code>, empty tuples <code>(,)</code>, dictionaries <code>{}</code> and sets <code>set()</code> are all <code>False</code> but are not equal to <code>None</code>.
 
Two different <code>None</code> instances are equal, because only one instance of the <code>None</code> object exists:
 
<syntaxhighlight lang='python'>
a = None
b = None
assert a == b
assert None == None # this equality statement is singled out as static validation warning
</syntaxhighlight>
 
===<span id='Boolean'></span>Booleans===
{{Internal|Python Boolean|Python Boolean}}
 
===Numbers===
====Integers====
{{Internal|Python Integers|Python Integers}}
 
====Floating Point Numbers====
Double-precision floating-point number. Note there is no separate double type.
<syntaxhighlight lang='python'>
x = 98.6
type(x)
<class 'float'>
</syntaxhighlight>
 
Floats can be used with operators (<code>+</code>, <code>-</code>, <code>*</code>, <code>/</code>, <code>//</code>, <code>**</code>, <code>%</code> and <code>divmod()</code> function).
=====NaN=====
NaN, standing for "Not a Number", is a special floating-point value that represents missing or undefined values in Python. NaN values can be tested with <code>math.isnan()</code> Python function or [[Pandas]] <code>isnull()</code> function. To handle NaN values, you can use Pandas <code>fillna()</code> function or <code>dropna()</code> function.
 
===Sequence Types===
There are three kinds of sequence types: [[Python_Language_String|strings]], [[Python_Language_Tuple#Overview|tuples]] and [[Python_Language_List#Overview|lists]]. Both contain zero or more elements, in both cases elements can be of different types. Strings and tuples are immutable. Lists are mutable. Mutability matters when the objects are stored in sets or as dictionary keys, because the collections are hashed in that case. If hashing a collection is not a concern, the rule of thumb is that fixed-size records of different objects are best represented as tuples, while variable-size collections of similar objects are best represented as lists.
====String====
{{Internal|Python_Language_String#Overview|String}}
====List====
{{Internal|Python Language List#Overview|List}}
====Tuple====
{{Internal|Python Language Tuple#Overview|Tuple}}
 
===Dictionary===
{{Internal|Python Language Dictionary#Overview|Dictionary}}
===Set===
{{Internal|Python Language Set#Overview|Set}}
===Dataclass===
{{Internal|Python_Dataclass#Overview|Dataclass}}
 
===Function===
{{Internal|Python Language Functions#Overview|Functions}}
===Built-in Types that Aren't Directly Accessible as a Builtin===
 
See <code>types.py</code>:
<syntaxhighlight lang='py'>
import types
 
print(types.FunctionType)
</syntaxhighlight>
===<tt>bytes</tt>===
Raw binary data.
 
==Checking the Type of an Object==
{{Internal|Python_Language_OOP#Type_Checking|Python Language OOP &#124; Type Checking}}
 
==<span id='Type_Conversions'></span>Type Conversions (Casting)==
===Explicit Type Conversions===
The built-in functions <code>[[#str()|str()]]</code>, <code>[[#int()|int()]]</code>, <code>[[#float()|float()]]</code> and <code>[[#bool()|bool()]]</code> can be used to cast values to those types:
There are built-in function that can be used for type conversion:
====<tt>str()</tt>====
{{Internal|Python_Language_String#Convert_other_Data_Types_to_Strings_with_str.28.29|Convert other Data Types to Strings with <tt>str()</tt>}}
====<tt>int()</tt>====
<code>int()</code> can be called on a boolean, float or on a string. For a boolean, <code>int(True)</code> will return 1 and <code>int(False)</code> will return 0.
 
If the string <code>int()</code> is invoked on cannot be converted to an integer, the function invocation throws a <code>ValueException</code>.
====<tt>float()</tt>====
<syntaxhighlight lang='py'>
>>> float('1.0e4')
10000.0
</syntaxhighlight>
====<tt>bool()</tt>====
 
===Implicit Type Conversions===
If numeric types are mixed, Python will automatically convert them:
<syntaxhighlight lang='py'>
>>> 4 + 7.0
11.0
</syntaxhighlight>
 
=Data Structures=
* [[Python_Safely_Navigate_a_Complex_Data_Structure#Overview|Safely Navigate a Complex Data Structure]]
 
A Python data structure is, for example, what you get when you parse a JSON-serialized text and you recreate the lists and the maps in memory.
==<span id='Collection'></span>Collections==
Organizatorium:  [[Python Module collections]]
 
==Iterable Types==
Iterable types: [[Python_Language_String#Overview|string]], [[Python Language List#Overview|list]], [[Python Language Tuple#Overview|tuple]], [[Python Language Set#Overview|set]].
 
===How to Tell whether an Instance is Iterable or Not===
 
Method 1:
 
<syntaxhighlight lang='py'>
if type(i) is str or type(i) is list or type(i) is tuple or type(i) is set:
    ...
</syntaxhighlight>
Method 2:
<syntaxhighlight lang='py'>
try:
    iter(i)
except TypeError:
    print('not iterable')
</syntaxhighlight>


=Statements=
===Iterator===
{{Internal|Python Iterators#Overview|Python Iterators}}
 
===Iterate Multiple Sequences with <tt>zip()</tt>===
<font color=darkkhaki>[[IPy]] Iterate Multiple Sequences with zip() Page 83.</font>
===Iterate over Code Structures with <tt>itertools</tt>===
<font color=darkkhaki>[[IPy]] Iterate over Code Structures with itertools Page 121.</font>
 
==Comprehensions==
{{Internal|Python Comprehensions#Overview|Comprehensions}}
 
=<span id='Statement'></span>Statements=
In Python 2, <code>print</code> used to be a statement, while in Python 3, <code>print()</code> is a [[#Functions|function]].
In Python 2, <code>print</code> used to be a statement, while in Python 3, <code>print()</code> is a [[#Functions|function]].
==Assignment Statement==
==Assignment Statement==
The assignment statement assigns a value to a [[#Variables|variable]].
The assignment statement assigns a value to a [[#Variables|variable]]. The assignment does not copy the value, it just attaches the variable name to the object that contains the data.
<syntaxhighlight lang='python'>
<syntaxhighlight lang='python'>
x = 1
x = 1
Line 37: Line 424:
<syntaxhighlight lang='python'>
<syntaxhighlight lang='python'>
x = x + 1
x = x + 1
</syntaxhighlight>
==Import Statement==
See: {{Internal|Python Language Modularization#Import|<tt>import</tt> Statement}}
==<span id='pass'></span>Statement that Does Nothing==
<code>pass</code> is a statement that indicates a function does nothing:
<syntaxhighlight lang='python'>
def do_nothing():
  pass
</syntaxhighlight>
</syntaxhighlight>


Line 42: Line 438:
Numeric expressions. Order of evaluation takes into account [[#Operator_Precedence|operator precedence]].  
Numeric expressions. Order of evaluation takes into account [[#Operator_Precedence|operator precedence]].  
==Operators==
==Operators==
{|class="wikitable" style="text-align: center; color: green;"
{|class="wikitable" style="text-align: left;"
| <code>+</code> || Addition || For numbers, adds them together, for strings, it concatenates.
| + || Addition || For numbers, adds them together, for strings, it concatenates. <code>+</code> can be combined with the assignment operator: <code>+=</code>
|-
|-
| <code>-</code> || Subtraction
| - || Subtraction || <code>-</code> can be combined with the assignment operator: <code>-=</code>
|-
|-
| <code>*</code> || Multiplication
| * || Multiplication || <code>*</code> can be combined with the assignment operator: <code>*=</code>
|-
|-
| <code>/</code> || Division
| / || Floating Point Division || In Python 3 integer division converts to floating point (not the case in Python 2, which truncates). <code>/</code> can be combined with the assignment operator: <code>/=</code>
|-
|-
| <code>**</code> || Power
| // || Integer (truncating) Division
|-
|-
| <code>%</code> || Remainder (modulo)
| ** || Power (exponentiation)
|-
| % || Remainder (modulo)
|-
| = || [[#Assignment_Statement|Assignment]] || Expression on the right side of = is calculated first, then assigned to the variable on the left side. See [[#Assignment_Statement|Assignment Statement]].
|-
| < || Less than
|-
| <= || Less than or Equal to
|-
| == || Equal to || Applies to strings, also. It is the mathematical equality. Also see [[#is|is]], [[#is_not|is not]] and [[#Identity,_Equality,_Comparability|Identity, Equality, Comparability]].
|-
| >= || Greater than or Equal to
|-
| > || Greater than
|-
| != || Not equal
|-
| <span id='is'></span>is || "is the same as" ||<code>is</code> keyword should be used to check if two variables refer to the same object. Returns a True or a False. Can be used in logical expression, implies "is the same as". It is similar but a stronger equality than "==". You should not use "is" when you should be using "==". "is" usually applies to [[#Boolean|True]], [[#Boolean|False]] or [[#None|None]]. Also see [[#Identity,_Equality,_Comparability|Identity, Equality, Comparability]].
|-
| <span id='is_not'></span>is not || "is not the same as" ||Returns a True or a False
|-
| in || The [[#Membership_with_in|membership operator]] ||
|-
| <span id=':='></span><tt>:=</tt> || The "walrus operator", introduced in Python 3.8. || Assigns values to variables as part of a larger expression: <code>if something_else := something == 'blue': [...]</code>
|-
|}
===Multiple Comparisons in a Single Expression===
<syntaxhighlight lang='py'>
x = 10
y = 20
assert 0 < x < y < 30
</syntaxhighlight>
===Boolean (Logical) Operators===
{|class="wikitable" style="text-align: left;"
| <span id='and'></span><tt>and</tt> || Logical AND
|-
| <span id='or'></span><tt>or</tt> || Logical OR
|-
| <span id='not'></span><tt>not</tt> || Logical NOT
|-
|-
| <code>=</code> ||
|}
|}


===Membership with <tt>in</tt>===
Membership in a [[#Collection|collection]] can be checked with the <code>in</code> operator. In case of a dictionary, <code>in</code> check the existence of a key:
<syntaxhighlight lang='py'>
str = 'abc'
list = ['a', 'b', 'c']
tu = ('a', 'b', 'c')
dict = {'a':'A', 'b':'B', 'c':'C'}
st = set()
st.add('a')
st.add('b')
assert 'a' in str
assert 'a' in list
assert 'a' in tu
assert 'a' in dict
assert 'a' in st
</syntaxhighlight>
The membership operator is commonly used in [[#for|<code>for</code> loops]].
==Ternary Operator==
{{External|https://book.pythontips.com/en/latest/ternary_operators.html}}
<syntaxhighlight lang='py'>
value_if_true if condition else value_if_false
</syntaxhighlight>
==Operator Precedence==
==Operator Precedence==
The following rules apply, and they are specified in the order of their descending precedence:
The following rules apply, and they are specified in the order of their descending precedence:
* Parentheses are always respected.
* Parentheses are always respected.
* Exponentiation
* Exponentiation.
* Multiplication, division and remainder
* Multiplication, division and remainder.
* Addition and subtraction
* Addition and subtraction.
* For operators with the same precedence, proceed left to right
* For operators with the same precedence, proceed left to right.


=Functions=
=<span id='Flow_Control'></span>Control Flow=
=Identifiers=
''We can solve problems in a way far more easily with clever data structures than with clever control flow. Control flow is obvious and data structures are subtle. So by making clever data structures, your control flow is simplified'' Dr. Charles Severance.
=Python Script=
A Python program file is called a Python script - a stored set of instructions that can be handed over to the Python interpreter. Python scripts have the .py extensions.
=Flow Control=
==Sequential Steps==
==Sequential Steps==
Sequential steps have the same indentation level.
Sequential steps have the same indentation level. A block with the same indentation level (recommended 4 spaces) designates a set of steps that execute sequentially. A colon <code>:</code> denotes the start of an indented code block after which all of the code must be indented by the same amount until the end of the block.


==Conditional Steps==
==Conditional Steps==
<code>if</code>, <code>elif</code> and <code>else</code>. are Python [[#Statement|statements]] that check whether a condition is True.
<font size=-1>
if ''[[#Expressions|expression]]'':
  ...
elif ''[[#Expressions|expression]]'':
  ...
else:
  ...
</font>
<syntaxhighlight lang='python'>
<syntaxhighlight lang='python'>
if x < 10:
if x < 10:
   print('something')
   print('something')
</syntaxhighlight>
</syntaxhighlight>
==Loops==
<syntaxhighlight lang='python'>
if a == 1:
  print('something')
else:
  print('something else')
</syntaxhighlight>
In the following case, once one of the alternative is triggered, the corresponding block is the '''only one''' that is executed, and the control gets out of the <code>if</code> statement. <code>else</code> is optional.
<syntaxhighlight lang='python'>
if a < 0:
  print('m')
elif a < 10:
  print('n')
elif a < 20:
  print('p')
else:
  print('q')
</syntaxhighlight>
 
==<span id='Loops'><span>Loops and Iterations==
===Indefinite Loops===
====<tt>while</tt>====
<syntaxhighlight lang='py'>
while <condition>:
  code-block
</syntaxhighlight>
<syntaxhighlight lang='python'>
<syntaxhighlight lang='python'>
n = 5
n = 5
Line 86: Line 574:
   n = n - 1
   n = n - 1
</syntaxhighlight>
</syntaxhighlight>
Loops have iteration [[#Variables|variables]], which are initialized, checked and changed within the loop.
<syntaxhighlight lang='py'>
while x < 5:
  x = x + 1
  print
</syntaxhighlight>
Loops have iteration [[#Variables|variables]], which are initialized, checked and changed within the loop. If the iteration variable that matters does not change within the loop, the loop will run forever - an infinite loop.
<syntaxhighlight lang='py'>
for i in range(5):
  print(i)
</syntaxhighlight>
===Definite Loops===
====<tt>for</tt>====
A <code>for</code> loop is finite, it goes through all elements of a collection: all the lines in a file, all the items in a list, all the characters in a string, all the keys in a dictionary, etc. As part of the <code>for</code> syntax, the iteration variable follows the [[#Reserved_Words|reserved word]] <code>for</code>, which is followed by the reserved word and [[#Membership_with_in|operator <code>in</code>]], which is then followed by a collection, which can be declared in-line or using a previously declared variable. The iteration variable iterates through the sequence (ordered set) and takes, in order, each value in the sequence. The statements to be executed in the loop are part of an [[#indented_block|indented block]]. The body is executed once for each value in the sequence.
 
<syntaxhighlight lang='py'>
for var_name in <collection>:
  code-block
</syntaxhighlight>
<syntaxhighlight lang='py'>
for i in [1, 2, 3, 4, 5]:
  print(i)
</syntaxhighlight>
<syntaxhighlight lang='py'>
collection = ['a', 'b', 'c']
for i in collection:
    print(i)
</syntaxhighlight>
⚠️ is <code>collection</code> is None, then the statement will fail with <code>TypeError</code>:
<syntaxhighlight lang='text'>
TypeError: 'NoneType' object is not iterable
</syntaxhighlight>
In case of the dictionary, the iteration is done over the dictionary's keys (or its <code>keys()</code> function). For more details about iterating over a dictionary's keys, values and both, see:
{{Internal|Python_Language_Dictionary#Iterate_over_a_Dictionary|Iterate over a Dictionary}}
<code>for</code> can also be used to [[#Iterate_Multiple_Sequences_with_zip.28.29|iterate over multiple sequences at the same time]], using the <code>zip()</code> function.
 
===Other Loop Statements===
 
====<tt>break</tt>====
<code>break</code> is a [[#Reserved_Words|reserved word]] that indicates a statement which breaks out of the innermost loop. When encountered, the execution goes to the first statement after the loop.
 
Python has a syntactical oddity that allows checking whether a <code>while</code> or a <code>for</code> loop '''did not''' exit with break:
<syntaxhighlight lang='py'>
i = 0
while i < 5:
  if i == 7:
    break
  i += 1
else:
  print('we did not exit with break')
</syntaxhighlight>
 
====<tt>continue</tt>====
<code>continue</code> is a [[#Reserved_Words|reserved word]] that indicates a statement which skips the current iteration and starts the next iteration. The control goes to the top of the loop.
===Loop Idioms===
====<span id='enumerate.28.29'></span>Access the Index and the Element at the Same Time with enumerate()====
{{Internal|Python_Language_List#Access_the_Index_and_the_Element_at_the_Same_Time_with_enumerate.28.29|List &#124; Access the Index and the Element at the Same Time with enumerate()}}
 
====<span id='reversed.28.29'></span>Iterate over a List in Reversed Order with reversed()====
{{Internal|Python_Language_List#Iterate_over_a_List_in_Reversed_Order|List &#124; Iterate over a List in Reversed Order}}
====Adding a Comma after All but Last Element====
<syntaxhighlight lang='py'>
l = ['a', 'b', 'c']
s = ', '.join(l)
assert 'a, b, c' == s
</syntaxhighlight>
If more complex processing is needed for each element, us a [[Python_Comprehensions#List_Comprehensions|list comprehension]].


==Functions==
==Functions==
Also handles notions like call stack, [[Python Language Functions#Frame|frame]], etc.
{{Internal|Python Language Functions|Functions}}
==<span id='try.2Fexcept'></span><span id='try'></span><span id='except'></span>Exceptions (<tt>try</tt>/<tt>except</tt>)==
{{Internal|Python Language Exceptions#Overview|Exceptions}}
=Generate Number Sequences with <tt>range()</tt>=
<font color=darkkhaki>[[IPy]] Generate Number Sequences with range() Page 83.</font>
<code>range(start, stop, step)</code>


=Classes=
<syntaxhighlight lang='py'>
for i in range(4, 7)
  print(i)
</syntaxhighlight>
prints:
<syntaxhighlight lang='text'>
4
5
6
</syntaxhighlight>
To count backwards:
<syntaxhighlight lang='py'>
for i in range(7, 4, -1):
  print(i)
</syntaxhighlight>
prints:
<syntaxhighlight lang='text'>
7
6
5
</syntaxhighlight>
 
=Generators=
{{Internal|Python_Generators#Overview|Generators}}
 
=Decorators=
{{Internal|Python_Decorators#Overview|Decorators}}
=Coroutines=
{{Internal|Python_Coroutines#Overview|Coroutines}}
 
=Traceback=
This means Python quit somewhere.
 
=<span id='Python_Script'></span><span id='Top-Level_Script'></span><span id='Module'></span><span id='Package'></span><span id='PyPI'></span>Modularization=
Discusses [[Python_Language_Modularization#Standalone_Program|standalone programs]], [[Python_Language_Modularization#Script|scripts]], [[Python_Language_Modularization#Module|modules]], [[Python_Language_Modularization#Package|packages]], [[Python_Language_Modularization#Import|importing]], [[Python_Language_Modularization#Package_Metadata|package metadata]]:
{{Internal|Python Language Modularization|Modularization}}
=Python Standard Library=
Python comes with a large standard library of [[Python_Language_Modularization#Module|modules]] that perform many useful tasks. This functionality is kept separate from the core language, to avoid bloating. However, they are shipped as part of the Python installation, so they are available locally wherever the Python runtime is installed. Because they the modules are shipped as part of the Python Standard Library, they are sometimes referred to as "standard libraries". When you are about to write some code, it's often worthwhile to first check whether there is a standard module that already does what you want.
 
The authoritative documentation for the modules included in the standard library is available here: {{External|[https://docs.python.org/3/library/ The Python Standard Library: https://docs.python.org/3/library/]}}
An introduction to the standard library is provided by this tutorial: {{External|[https://docs.python.org/3.3/tutorial/stdlib.html Brief Tour of the Standard Library: https://docs.python.org/3.3/tutorial/stdlib.html]}}
More documentation on the standard library is available here: {{External|[https://pymotw.com/2/contents.html Doug Hellmann's Python Module of the Week: https://pymotw.com/2/contents.html]}}
==Python Module Index==
The index of the modules shipped as part of the standard library is available here: {{External|[https://docs.python.org/3/py-modindex.html Python Module Index https://docs.python.org/3/py-modindex.html]}}
==Notable Python Standard Library Modules==
* <span id='json'></span>[[Python_Module_json#Overview|json]]
* <span id='os'></span>[[Python_Module_os#Overview|os]]
* <span id='urllib'></span>[[Python_Package_urllib#Overview|urllib]]
* <span id='venv'></span>[[Python_Module_venv#Overview|venv]]
* <span id='webbrowser'></span>[[Python_Module_webbrowser#Overview|webbrowser]]
* <span id='shutil'></span>[[Python_Module_shutil#Overview|shutil]]
* <span id='sys'></span>[[Python_Module_sys#Overview|sys]]
* <span id='collections'></span>[[Python_Module_collections#Overview|collections]]
* <span id='re'></span>[[Python_Module_re#Overview|re]]
* <span id='logging'></span>[[Python_Module_logging#Overview|logging]]
* <span id='inspect'></span>[[Python_Introspection#The_inspect_Standard_Library_Module|inspect]]
* <span id='unittest'></span><span id='unittest.mock'></span>[[Python_Module_unittest#Overview|unittest]] and [[Python_Module_unittest#unittest.mock|unittest.mock]]
* <span id='uuid'></span>[[Python_Module_uuid#Overview|uuid]]
* doctest
* <span id='random'></span>[[Python_Module_ random#Overview|random]]
* <span id='configparser'></span>[[Python_Package_configparser#Overview|configparser]]
 
=<span id='PyPI'></span>Python Package Index PyPI=
The Python Package Index (PyPI) is a repository of software for the Python programming language. PyPI helps with finding and installing software developed and shared by the Python community.{{External|https://pypi.org}}
It can be searched with <code>[[Pip#search|pip search]]</code>.
==TestPyPI==
{{External|https://packaging.python.org/en/latest/guides/using-testpypi/}}
TestPyPI is a package index intended for testing and experimentation. You will need to register an account https://test.pypi.org/account/register/
==Notable Packages==
* <span id='argparse'></span>[[Python Package argparse#Overview|argparse]]
* <span id='PyYAML'></span>[[YAML_in_Python#PyYAML|PyYAML]]
* <span id='pytest'></span>[[Pytest#Overview|pytest]]
* <span id='deprecated'></span>[[Python_Package_Deprecated#Overview|deprecated]]
* <span id='PyGithub'></span>[[Python_Package_PyGithub#Overview|PyGithub]]
* <span id='GitPython'></span>[[Python_Package_GitPython#Overview|GitPython]]
* <span id='semantic-version'></span>[[Python Package semantic-version|semantic-version]]
* <span id='semver'></span>[[Python_Package_semver|semver]]
* <span id='kubernetes'></span>[[Python_Package_kubernetes|kubernetes]]
* <span id='awscli'></span>[[Python_Package_awscli|awscli]]
* <span id='boto3'></span>[[Python_Package_boto3|boto3]]
* requests https://pypi.org/project/requests/
 
=Object-Oriented Programming=
{{Internal|Python_Language_OOP#Overview|Classes and Objects}}
 
=<span id='Environment'></span><span id='Activated_Virtual_Environment_Shell'></span><span id='Bash_Wrapper_that_Bootstraps_a_Virtual_Environment'></span>Virtual Environment=
{{Internal|Python Virtual Environment#Overview|Python Virtual Environment}}
 
=Python Enhancement Proposals (PEPs)=
=Special Names=
====<tt>__main__</tt>====
{{External|https://docs.python.org/3/library/__main__.html}}
Also see: {{Internal|Python_Language_Modularization#Distinguishing_between_Importing_and_Executing_a_Module|Distinguishing between Importing and Executing a Module}}
 
=Layout and Structure of a Python Project=
{{Internal|Layout of a Python Project#Overview|Layout of a Python Project}}
=Dunder=
Generic appellative given to the names enclosed within "__", for example "__main__".
=<span id='Thread'></span>Threads and Concurrency=
{{Internal|Python Threads and Concurrency#Overview|Threads and Concurrency}}
=<span id='Context_Manager'></span><tt>with</tt> and Context Manager=
{{Internal|Python_Context_Manager#Overview|Context Manager}}
 
=Protocol=
{{External|https://peps.python.org/pep-0544/}}
=Identity, Equality, Comparability=
 
<code>is</code> is an operator that can be used to check if two variables refer to the same object. A common use of <code>is</code> and <code>is not</code> is to check if a variable is <code>None</code>, since there is only one instance of <code>None</code>. The <code>==</code> operator checks value equality:
 
<syntaxhighlight lang='py'>
a = [1, 2, 3]
b = [1, 2, 3]
assert a is not b
assert a == b
</syntaxhighlight>


=Organizatorium=
=Code Examples=
====Nesting====
{{Internal|Python Code Examples|Python Code Examples}}

Latest revision as of 23:08, 19 September 2024

External

Internal

TODO

Expand the following sections and subjects:

Overview

Python is a general-purpose, high-level, dynamically-typed language. Its design makes it very readable. Its relative terseness makes it possible to write a program that is much smaller than the equivalent static language program. However, if the program is CPU-bound, a program written in C, C++ or Java will generally run faster than its Python equivalent. While Python offers thread abstractions via libraries, only one thread can hold the control of the Python interpreter, because of the Global Interpreter Lock (GIL).

Python installation contains the core language support (built-ins) and standard libraries.

Python programs can be executed in two modes: interactively via an interpreter, also called a shell, or stored into a file with the usual, but optional .py extension and run by typing python followed by the file name.

In Python, everything is an object. This includes numbers, strings, tuples, lists, dictionaries, functions and programs. The ID of any object can be obtained with the built-in function id(). The definition of an object is called class.

In Python, spacing does matter, sequential blocks are indented at the same level, and they are known as indented blocks. Python uses white space to define program structure.

Python Version

See:

Python Versions

Printing

Printing done with the print() function in Python 3 (it uses to be a statement in Python 2).

print('something')

Also see:

Printing to stdout in Python

Comments

# This is a comment
s = 0 # This is also a comment

Everything from # to the end of the line is comment. Python does not have a multi-line comment.

Code Line Continuation

Python encourages short lines. Recommended maximum line length is 80 characters. To continue a line on the next line, use backslash:

s = 'A' + \
 'B' + \
 'C'

Line continuation is needed if a Python expression spans multiple lines.

Reserved Words

Reserved words, or keywords, can only be used to mean the thing Python expects them to mean. They cannot be used as variable names, function names, class names, or identifiers.

try except finally raise assert
while for break continue in
False True None
if else elif
def return lambda
and or not
import from
with as
nonlocal yield
pass
is
del
class
global

Also see:

Keywords

The as Reserved Word

Used:

Literals

Literals have a type.

Integer Literals

String Literals

Strings | Quotes

F-String

F-String

Constants

Constants are fixed values, they do not change throughout the program. Constants can be boolean (True, False), numeric (integers or floating point numbers), or strings, which can be single quoted or double quoted, or even "the absence of a value" (None). Constants can be assigned to variables, can be arguments of functions. Constants have a type.

Constant Variable

Python does not have anything equivalent to final in Java, so any variable can be modified after assignment. You can use:

CONST_NAME = "Name"

for clarity, but nothing prevents CONST_NAME to be assigned other value later in the program.

Single Quoted, Double Quoted, Triple Single Quoted, Triple Double Quoted

TODO

Multi-Line Strings

TODO, see Python Language Functions#Docstring

Triple Double-Quoted Strings

Triple double-quoted strings is the recommended style for docstrings.

s = """
This is a
  multi-line
  string
"""

Variables

Variables are names associated with memory locations used to store values. Variables and identifiers are equivalent. Variables are declared and assigned a value through an assignment statement. Assignment is also referred to as binding, as we are binding a name to an object. Variable names that have been assigned may be occasionally referred to as "bound variables".

The assignment does not copy a value, it just attaches the name to the object that contains the data. See reference variables, below.

A variable may be associated with an object that has a type, and the same variable may be assigned later an object of a different type. Python variables do not have an inherent type associated with them, a variable can refer to a different type of object simply by doing an assignment. The type information is stored in the object itself. A useful mental representation of a variable is a sticky note that can be attached to an object, and then re-attached to a different object, not necessarily of the same type.

a = 1
b = 'something'
print(a)
print(b)
a = 'something else'
print(a)

Also see:

Variables, Parameters, Arguments

All Python Variables are Reference Variables

All Python variables are reference variables. When objects are passed as arguments to a function, new local variables are created on the function stack, referencing the original objects without any copying. For more details, see:

Variables, Parameters, Arguments | Reference

Variable Naming Rules

Variable names are case sensitive. Variable names can start with letters or underscore ('_') - but underscores should be generally avoided because Python treats names that begin with an underscore in special ways, and tends to use them for its internal purposes. The rest of the variable name can be lowercase letters (a through z), uppercase letters (A through Z), digits (0 through 9) and underscores. No other characters are allowed. Python has a set of words, called reserved words, that cannot be used as variable names. Variable names should be sensible (mnemonic). Function name follow the same rules.

Variables declared in functions should be lowercase. For more details see:

Python Style

Leading Underscore Variable Names

See:

Two Underscore (__) Variables

Names that being and end with two underscores (__) are reserved for use within Python, for built-in variables.

__name__

__name__ is a built-in variable which evaluates to the name of the current module and it is set differently depending on whether the module is imported or executed. For more details see:

Distinguishing between Importing and Executing a Module

__package__

https://docs.python.org/3/reference/import.html#package__
https://www.python.org/dev/peps/pep-0366

__package__ is a module-level attribute, which must be set. Its value must be a string, but it can be the same value as its __name__. When the module is a package, its __package__ value should be set to its __name__. When the module is not a package, __package__ should be set to the empty string for top-level modules, or for submodules, to the parent package’s name. When it is present, relative imports will be based on this attribute rather than the module __name__ attribute.

__file__

Contains the name of the file the function was loaded from.

__doc__

__loader__

__spec__

__annotations__

__builtins__

Variables Namespace and Scope

A namespace is a collection of names that can be used to access various underlying Python objects, such as data instances, functions, classes, etc. Multiple namespaces can co-exist at a given time, but they are isolated from each other. Within a namespace, a particular name is unique. A Python program has three kinds of namespaces: built-in, global and local.

Python Namespaces.png

A module also has a global namespace.

A scope is the portion of a program where a namespace can be accessed directly without any prefix. At any given moment, there are at least three nested scopes: the scope fo the current function that has local names, the scope of the module, which has global names and the outermost scope that has built-in names. When a reference is made inside a function, the name is searched in the local namespace, the global namespace and finally in the built-in namespace. If there is a function inside another function, the embedded function's local scope is nested in the upper level function's local scope.

The variables declared in higher scopes can be read, without any prefix, but unless they are prefixed with the namespace they belong to, they cannot be modified. Only the variables in the same scope can be modified. If we try to modify a variable from a higher scope without prefixing it (which we can read without prefixing it), a local variable with the same name is created and that variable is written. For more details see Interaction between Local and Global Variables, below.

Built-in Namespace

The built-in namespace contains all the built-in Python runtime names, such those of the built-in functions, for example. This namespace is create when the Python interpreter is started and exists for as long as the interpreter runs. The built-in namespace is the reason that built-in functions like id() and print() are always available in any part of the program, such a module, a function or a class.

Global Namespace and Variables

Each module creates its own global namespace for its variables, functions and classes. Since the namespaces are isolated, the same name may exist in different modules and it does not collide. The variables defined in the global namespace, outside any function or class, are known as global variables. Global variables can be used by everyone within the module, both inside and outside of functions. The global variables are maintained by Python in the module's global symbol table.

Global Symbol Table and globals()

The Python runtime maintains a global symbol table, which is a data structure that contains variable names and their association with objects in the global namespace of a module. The global symbol table contains all variables that are not associated with any class or function. The built-in function globals() returns a dictionary of the contents of the global symbol table. Assuming that the main program contains a some_func function declaration and a SomeClass class declaration, the content of the global symbol table of the corresponding module is similar to:

__name__        = '__main__'
__doc__         = None
__package__     = None
__loader__      = <_frozen_importlib_external.SourceFileLoader object at 0x1090d5fa0>
__spec__        = None
__annotations__ = {}
__builtins__    = <module 'builtins' (built-in)>
__file__        = './main.py'
__cached__      = None
some_func       = <function some_func at 0x101dd4310>
SomeClass       = <class '__main__.SomeClass'>

Local Namespaces and Variables

Each function and class define their own namespace, which is created when the function is called or the class is instantiated. Variables defined inside a function are distinct from the variables with the same name defined in the main program, and they are known as local variables. However, the global variables are accessible within functions. Upon access from a function, the global variable with the same name will remain as it was, global and with the original value. Special rules apply when a local variable has the same name as an existing global variable. In this case we say that the local variable temporarily overrides the global variable, and this interaction is described in the Interaction between Local and Global Variables section.

Local Symbol Table and locals()

Each function and class contains a local symbol table, which maintains the names of the variables in the local namespace. The built-in function locals() returns a dictionary of the contents of the local namespace.

Get Names in a Namespace with dir()

The built-in function dir() returns a list of defined name in a namespace. Without arguments, it produces an alphabetically sorted list of names in the current local symbol table. When given an argument that is the name of an imported module, dir() lists the names defined in the imported module.

Interaction between Local and Global Variables

If a global variable is accessed inside a function, and then there is an attempt to change it, the program throws an UnboundLocalError:

a = 10

def some_func():
    print('a:', a)
    a = 11

some_func() # this throws UnboundLocalError
Traceback (most recent call last):
  File "./main.py", line 8, in <module>
    some_func()
  File "./main.py", line 5, in some_func
    print('a:', a)
UnboundLocalError: local variable 'a' referenced before assignment

If a variable with the same name as a global variable is modified without accessing the global variable first, this implicitly declares the local variable with the same name, and the program works fine:

a = 10

def some_func():
    a = 11
    print('a:', a)

some_func()

This will display:

a: 11

To explicitly indicate that we want to use the global variable inside the function, use the global reserved word. This will tell the interpreter that we want to use the global variable, which then can be modified inside the function:

a = 10

def some_func():
    global a
    print("global 'a' from within the function:", a)
    a = 11

some_func()
print("global 'a' after function call:", a)

If global is not explicitly declared inside a function, Python uses the local namespace and the variable is implicitly local. Local variables are created during the function execution and discarded after the function completes.

global keyword use is discouraged. Typically, global variables are used to store some kind of state in a system. If you find yourself using a lot of them, it may indicate a need for object-oriented programming.

nonlocal

TODO nonlocal.

Identifiers

An identifier is a name given to objects. Everything in Python is an object, so the name is the way to access the underlying object. An identifier and a variable are equivalent.

Type

Python is a dynamically typed language, in that the same variable may be bound to an object of a certain type, then to another object of a different type, at different moments in the execution of the program. Everything in Python is implemented as an object, and any object has a type, but the type information is stored in the object, not in the variable. The type determines whether the data value of the object is mutable or immutable (constant).

Python is strongly typed, which means that the type of an object does not change, even if the value is mutable. Strong typing is a property of the object instances, not of variables that are bound to those objects.

The type of an object can be obtained with the built-in function type() applied to the variable or a constant the object is assigned to. In Python, "class" and "type" mean pretty much the same thing.

Data Types

Scalar types, or scalars, are "single value" types: bool, numbers.

None

x = None
type(x)
<class 'NoneType'>

None is a special Python value that holds a place when there is nothing to say. It is not the same as False, although it looks false when it is evaluated as a boolean. None can be used with the is or is not operators:

if x is None:
  ...
if x is not None:
  ...

None is returned by a function that does not contain the return statement. None is useful to distinguish a missing value from an empty value. Zero-value integers or floats, empty strings (''), empty lists [], empty tuples (,), dictionaries {} and sets set() are all False but are not equal to None.

Two different None instances are equal, because only one instance of the None object exists:

a = None
b = None
assert a == b
assert None == None # this equality statement is singled out as static validation warning

Booleans

Python Boolean

Numbers

Integers

Python Integers

Floating Point Numbers

Double-precision floating-point number. Note there is no separate double type.

x = 98.6
type(x)
<class 'float'>

Floats can be used with operators (+, -, *, /, //, **, % and divmod() function).

NaN

NaN, standing for "Not a Number", is a special floating-point value that represents missing or undefined values in Python. NaN values can be tested with math.isnan() Python function or Pandas isnull() function. To handle NaN values, you can use Pandas fillna() function or dropna() function.

Sequence Types

There are three kinds of sequence types: strings, tuples and lists. Both contain zero or more elements, in both cases elements can be of different types. Strings and tuples are immutable. Lists are mutable. Mutability matters when the objects are stored in sets or as dictionary keys, because the collections are hashed in that case. If hashing a collection is not a concern, the rule of thumb is that fixed-size records of different objects are best represented as tuples, while variable-size collections of similar objects are best represented as lists.

String

String

List

List

Tuple

Tuple

Dictionary

Dictionary

Set

Set

Dataclass

Dataclass

Function

Functions

Built-in Types that Aren't Directly Accessible as a Builtin

See types.py:

import types

print(types.FunctionType)

bytes

Raw binary data.

Checking the Type of an Object

Python Language OOP | Type Checking

Type Conversions (Casting)

Explicit Type Conversions

The built-in functions str(), int(), float() and bool() can be used to cast values to those types: There are built-in function that can be used for type conversion:

str()

Convert other Data Types to Strings with str()

int()

int() can be called on a boolean, float or on a string. For a boolean, int(True) will return 1 and int(False) will return 0.

If the string int() is invoked on cannot be converted to an integer, the function invocation throws a ValueException.

float()

>>> float('1.0e4')
10000.0

bool()

Implicit Type Conversions

If numeric types are mixed, Python will automatically convert them:

>>> 4 + 7.0
11.0

Data Structures

A Python data structure is, for example, what you get when you parse a JSON-serialized text and you recreate the lists and the maps in memory.

Collections

Organizatorium: Python Module collections

Iterable Types

Iterable types: string, list, tuple, set.

How to Tell whether an Instance is Iterable or Not

Method 1:

if type(i) is str or type(i) is list or type(i) is tuple or type(i) is set:
    ...

Method 2:

try:
    iter(i)
except TypeError:
    print('not iterable')

Iterator

Python Iterators

Iterate Multiple Sequences with zip()

IPy Iterate Multiple Sequences with zip() Page 83.

Iterate over Code Structures with itertools

IPy Iterate over Code Structures with itertools Page 121.

Comprehensions

Comprehensions

Statements

In Python 2, print used to be a statement, while in Python 3, print() is a function.

Assignment Statement

The assignment statement assigns a value to a variable. The assignment does not copy the value, it just attaches the variable name to the object that contains the data.

x = 1

The assignment statement accepts expressions:

x = x + 1

Import Statement

See:

import Statement

Statement that Does Nothing

pass is a statement that indicates a function does nothing:

def do_nothing():
  pass

Expressions

Numeric expressions. Order of evaluation takes into account operator precedence.

Operators

+ Addition For numbers, adds them together, for strings, it concatenates. + can be combined with the assignment operator: +=
- Subtraction - can be combined with the assignment operator: -=
* Multiplication * can be combined with the assignment operator: *=
/ Floating Point Division In Python 3 integer division converts to floating point (not the case in Python 2, which truncates). / can be combined with the assignment operator: /=
// Integer (truncating) Division
** Power (exponentiation)
% Remainder (modulo)
= Assignment Expression on the right side of = is calculated first, then assigned to the variable on the left side. See Assignment Statement.
< Less than
<= Less than or Equal to
== Equal to Applies to strings, also. It is the mathematical equality. Also see is, is not and Identity, Equality, Comparability.
>= Greater than or Equal to
> Greater than
!= Not equal
is "is the same as" is keyword should be used to check if two variables refer to the same object. Returns a True or a False. Can be used in logical expression, implies "is the same as". It is similar but a stronger equality than "==". You should not use "is" when you should be using "==". "is" usually applies to True, False or None. Also see Identity, Equality, Comparability.
is not "is not the same as" Returns a True or a False
in The membership operator
:= The "walrus operator", introduced in Python 3.8. Assigns values to variables as part of a larger expression: if something_else := something == 'blue': [...]

Multiple Comparisons in a Single Expression

x = 10
y = 20
assert 0 < x < y < 30

Boolean (Logical) Operators

and Logical AND
or Logical OR
not Logical NOT

Membership with in

Membership in a collection can be checked with the in operator. In case of a dictionary, in check the existence of a key:

str = 'abc'
list = ['a', 'b', 'c']
tu = ('a', 'b', 'c')
dict = {'a':'A', 'b':'B', 'c':'C'}
st = set()
st.add('a')
st.add('b')
assert 'a' in str
assert 'a' in list
assert 'a' in tu
assert 'a' in dict
assert 'a' in st

The membership operator is commonly used in for loops.

Ternary Operator

https://book.pythontips.com/en/latest/ternary_operators.html
value_if_true if condition else value_if_false

Operator Precedence

The following rules apply, and they are specified in the order of their descending precedence:

  • Parentheses are always respected.
  • Exponentiation.
  • Multiplication, division and remainder.
  • Addition and subtraction.
  • For operators with the same precedence, proceed left to right.

Control Flow

We can solve problems in a way far more easily with clever data structures than with clever control flow. Control flow is obvious and data structures are subtle. So by making clever data structures, your control flow is simplified Dr. Charles Severance.

Sequential Steps

Sequential steps have the same indentation level. A block with the same indentation level (recommended 4 spaces) designates a set of steps that execute sequentially. A colon : denotes the start of an indented code block after which all of the code must be indented by the same amount until the end of the block.

Conditional Steps

if, elif and else. are Python statements that check whether a condition is True.

if expression:
  ...
elif expression:
  ...
else:
  ...

if x < 10:
  print('something')
if a == 1:
  print('something')
else:
  print('something else')

In the following case, once one of the alternative is triggered, the corresponding block is the only one that is executed, and the control gets out of the if statement. else is optional.

if a < 0:
  print('m')
elif a < 10:
  print('n')
elif a < 20:
  print('p')
else:
  print('q')

Loops and Iterations

Indefinite Loops

while

while <condition>:
  code-block
n = 5
while n > 0:
  print(n)
  n = n - 1
while x < 5:
  x = x + 1
  print

Loops have iteration variables, which are initialized, checked and changed within the loop. If the iteration variable that matters does not change within the loop, the loop will run forever - an infinite loop.

for i in range(5):
  print(i)

Definite Loops

for

A for loop is finite, it goes through all elements of a collection: all the lines in a file, all the items in a list, all the characters in a string, all the keys in a dictionary, etc. As part of the for syntax, the iteration variable follows the reserved word for, which is followed by the reserved word and operator in, which is then followed by a collection, which can be declared in-line or using a previously declared variable. The iteration variable iterates through the sequence (ordered set) and takes, in order, each value in the sequence. The statements to be executed in the loop are part of an indented block. The body is executed once for each value in the sequence.

for var_name in <collection>:
  code-block
for i in [1, 2, 3, 4, 5]:
  print(i)
collection = ['a', 'b', 'c']
for i in collection:
    print(i)

⚠️ is collection is None, then the statement will fail with TypeError:

TypeError: 'NoneType' object is not iterable

In case of the dictionary, the iteration is done over the dictionary's keys (or its keys() function). For more details about iterating over a dictionary's keys, values and both, see:

Iterate over a Dictionary

for can also be used to iterate over multiple sequences at the same time, using the zip() function.

Other Loop Statements

break

break is a reserved word that indicates a statement which breaks out of the innermost loop. When encountered, the execution goes to the first statement after the loop.

Python has a syntactical oddity that allows checking whether a while or a for loop did not exit with break:

i = 0
while i < 5:
  if i == 7:
    break
  i += 1
else:
  print('we did not exit with break')

continue

continue is a reserved word that indicates a statement which skips the current iteration and starts the next iteration. The control goes to the top of the loop.

Loop Idioms

Access the Index and the Element at the Same Time with enumerate()

List | Access the Index and the Element at the Same Time with enumerate()

Iterate over a List in Reversed Order with reversed()

List | Iterate over a List in Reversed Order

Adding a Comma after All but Last Element

l = ['a', 'b', 'c']
s = ', '.join(l)
assert 'a, b, c' == s

If more complex processing is needed for each element, us a list comprehension.

Functions

Also handles notions like call stack, frame, etc.

Functions

Exceptions (try/except)

Exceptions

Generate Number Sequences with range()

IPy Generate Number Sequences with range() Page 83.

range(start, stop, step)

for i in range(4, 7)
  print(i)

prints:

4
5
6

To count backwards:

for i in range(7, 4, -1):
  print(i)

prints:

7
6
5

Generators

Generators

Decorators

Decorators

Coroutines

Coroutines

Traceback

This means Python quit somewhere.

Modularization

Discusses standalone programs, scripts, modules, packages, importing, package metadata:

Modularization

Python Standard Library

Python comes with a large standard library of modules that perform many useful tasks. This functionality is kept separate from the core language, to avoid bloating. However, they are shipped as part of the Python installation, so they are available locally wherever the Python runtime is installed. Because they the modules are shipped as part of the Python Standard Library, they are sometimes referred to as "standard libraries". When you are about to write some code, it's often worthwhile to first check whether there is a standard module that already does what you want.

The authoritative documentation for the modules included in the standard library is available here:

The Python Standard Library: https://docs.python.org/3/library/

An introduction to the standard library is provided by this tutorial:

Brief Tour of the Standard Library: https://docs.python.org/3.3/tutorial/stdlib.html

More documentation on the standard library is available here:

Doug Hellmann's Python Module of the Week: https://pymotw.com/2/contents.html

Python Module Index

The index of the modules shipped as part of the standard library is available here:

Python Module Index https://docs.python.org/3/py-modindex.html

Notable Python Standard Library Modules

Python Package Index PyPI

The Python Package Index (PyPI) is a repository of software for the Python programming language. PyPI helps with finding and installing software developed and shared by the Python community.

https://pypi.org

It can be searched with pip search.

TestPyPI

https://packaging.python.org/en/latest/guides/using-testpypi/

TestPyPI is a package index intended for testing and experimentation. You will need to register an account https://test.pypi.org/account/register/

Notable Packages

Object-Oriented Programming

Classes and Objects

Virtual Environment

Python Virtual Environment

Python Enhancement Proposals (PEPs)

Special Names

__main__

https://docs.python.org/3/library/__main__.html

Also see:

Distinguishing between Importing and Executing a Module

Layout and Structure of a Python Project

Layout of a Python Project

Dunder

Generic appellative given to the names enclosed within "__", for example "__main__".

Threads and Concurrency

Threads and Concurrency

with and Context Manager

Context Manager

Protocol

https://peps.python.org/pep-0544/

Identity, Equality, Comparability

is is an operator that can be used to check if two variables refer to the same object. A common use of is and is not is to check if a variable is None, since there is only one instance of None. The == operator checks value equality:

a = [1, 2, 3]
b = [1, 2, 3]
assert a is not b
assert a == b

Code Examples

Python Code Examples