#Python
How to send GET or POST requests
How to send GET or POST requests
import requests
r = requests.get(url)
r = requests.post(url, data=data)
r = r.json()
Exceptions
Events that occur during the execution of a program and disrupt the normal flow. An exception is an object that represents an error.
When raised, exceptions must be handled immediately, otherwise they would terminate the execution. If you have some suspicious code that may raise an exception, you can defend your program by placing it in a try block, followed by a except block which handles the problem as elegantly as possible.
The list of all #Python exceptions:
https://docs.python.org/3/library/exceptions.html
Events that occur during the execution of a program and disrupt the normal flow. An exception is an object that represents an error.
When raised, exceptions must be handled immediately, otherwise they would terminate the execution. If you have some suspicious code that may raise an exception, you can defend your program by placing it in a try block, followed by a except block which handles the problem as elegantly as possible.
try:
You do your operations here;
......................
except ExceptionI:
If there is ExceptionI, then execute this block.
except ExceptionII:
If there is ExceptionII, then execute this block.
......................
else:
If there is no exception then execute this block.
The list of all #Python exceptions:
https://docs.python.org/3/library/exceptions.html
The diamond problem
- is an ambiguity that arises when two classes B and C inherit from A, and class D inherits from both B and C. If there is a method in A that B and C have overridden, and D does not override it, then which version of the method does D inherit: that of B, or that of C?
For example, in the context of GUI software development, a class Button may inherit from both classes Rectangle (for appearance) and Clickable (for functionality/input handling), and classes Rectangle and Clickable both inherit from the Object class. Now if the equals method is called for a Button object and there is no such method in the Button class but there is an overridden equals method in Rectangle or Clickable (or both), which method should be eventually called?
How different languages deal with it:
• Common #Lisp: by order "...in the order in which parent classes are named in the subclass definition"
• Curl: "and the secondary constructor will be invoked for all other subclasses."
• Eiffel: "Eiffel will automatically join features together, if they have the same name and implementation."
• Go: compile-time error
• #Java: compile-time error
• #OCaml: by order "...are inherited in the same order, with each newly inherited method overriding any existing methods."
• #Perl: by order "...from as an ordered list. The compiler uses the first method it finds..."
• #Python: by order
• #Ruby: by order "...as rightmost depth first resolution."
• #Scala: by order "allows multiple instantiation of traits, which allows for multiple inheritance by adding a distinction between the class hierarchy and the trait hierarchy. A class can only inherit from a single class, but can mix-in as many traits as desired." This approach is the most similar to #Nile's; traits being qualifications.
• Tcl: by order "the order of specification in the class declaration affects the name resolution for members..."
Languages that allow only single inheritance, where a class can only derive from one base class, do not have the diamond problem.
Moreover, languages such as #Ada, Objective-C, #CSharp, #Delphi/Free #Pascal, Java, #Swift and PHP allow multiple-inheritance of interfaces (called protocols in Objective-C and Swift). Interfaces are like abstract base classes that specify method signatures without implementing any behavior.
When several interfaces declare the same method signature, as soon as that method is implemented (defined) anywhere in the inheritance chain, it overrides any implementation of that method in the chain above it (in its superclasses). Hence, at any given level in the inheritance chain, there can be at most one implementation of any method. Thus, single-inheritance method implementation does not exhibit the Diamond Problem even with multiple-inheritance of interfaces.
- is an ambiguity that arises when two classes B and C inherit from A, and class D inherits from both B and C. If there is a method in A that B and C have overridden, and D does not override it, then which version of the method does D inherit: that of B, or that of C?
For example, in the context of GUI software development, a class Button may inherit from both classes Rectangle (for appearance) and Clickable (for functionality/input handling), and classes Rectangle and Clickable both inherit from the Object class. Now if the equals method is called for a Button object and there is no such method in the Button class but there is an overridden equals method in Rectangle or Clickable (or both), which method should be eventually called?
How different languages deal with it:
• Common #Lisp: by order "...in the order in which parent classes are named in the subclass definition"
• Curl: "and the secondary constructor will be invoked for all other subclasses."
• Eiffel: "Eiffel will automatically join features together, if they have the same name and implementation."
• Go: compile-time error
• #Java: compile-time error
• #OCaml: by order "...are inherited in the same order, with each newly inherited method overriding any existing methods."
• #Perl: by order "...from as an ordered list. The compiler uses the first method it finds..."
• #Python: by order
• #Ruby: by order "...as rightmost depth first resolution."
• #Scala: by order "allows multiple instantiation of traits, which allows for multiple inheritance by adding a distinction between the class hierarchy and the trait hierarchy. A class can only inherit from a single class, but can mix-in as many traits as desired." This approach is the most similar to #Nile's; traits being qualifications.
• Tcl: by order "the order of specification in the class declaration affects the name resolution for members..."
Languages that allow only single inheritance, where a class can only derive from one base class, do not have the diamond problem.
Moreover, languages such as #Ada, Objective-C, #CSharp, #Delphi/Free #Pascal, Java, #Swift and PHP allow multiple-inheritance of interfaces (called protocols in Objective-C and Swift). Interfaces are like abstract base classes that specify method signatures without implementing any behavior.
When several interfaces declare the same method signature, as soon as that method is implemented (defined) anywhere in the inheritance chain, it overrides any implementation of that method in the chain above it (in its superclasses). Hence, at any given level in the inheritance chain, there can be at most one implementation of any method. Thus, single-inheritance method implementation does not exhibit the Diamond Problem even with multiple-inheritance of interfaces.
#TalkingSemicolon
To use all/any, you need to construct a new iterator. You can override their behavior:
#Python
To use all/any, you need to construct a new iterator. You can override their behavior:
def __and__(self, other): return Falsewhich also affects
all.#Python
#TalkingSemicolon
Tools for natural language generation
Either Keras/TensorFlow if going with #Python or TorchNN if going with #Lua.
Nothing else really works.
Tools for natural language generation
Either Keras/TensorFlow if going with #Python or TorchNN if going with #Lua.
Nothing else really works.
Blocks
- are lexical structures that allow many statements to be treated as one. A language that allows blocks and nested blocks, is called block-structured. Blocks are fundamental to #structured programming.
As scopes
Depending on the language, certain distinguished blocks may be treated as lexical scopes; otherwise, identifiers assigned in outer blocks are visible inside inner blocks, unless shadowed.
Syntax
• Free-form
Whitespace only delimits tokens and has no other significance
•
•
•
• Off-side rule
Indentation groups blocks of code
e.g. #Python, #Haskell, #Cobra, #CoffeeScript
Limitations
In some languages blocks do not fully support all declarations; for instance many C-derived languages do not permit nested functions.
- are lexical structures that allow many statements to be treated as one. A language that allows blocks and nested blocks, is called block-structured. Blocks are fundamental to #structured programming.
As scopes
Depending on the language, certain distinguished blocks may be treated as lexical scopes; otherwise, identifiers assigned in outer blocks are visible inside inner blocks, unless shadowed.
Syntax
• Free-form
Whitespace only delimits tokens and has no other significance
•
begin ... end: #ALGOL, #Pascal•
{ ... }: #C, #Perl, #JS, #Nile•
( keyword ... ): #Lisp• Off-side rule
Indentation groups blocks of code
e.g. #Python, #Haskell, #Cobra, #CoffeeScript
Limitations
In some languages blocks do not fully support all declarations; for instance many C-derived languages do not permit nested functions.
Data layout
- is how multidimensional arrays are stored in a linear storage such as RAM. It is critical for:
• correctly passing arrays between programs written in different programming languages
• performance when traversing an array because modern CPUs, due to caching, process sequential data more efficiently than non-sequential data
• contiguous access makes it possible to use SIMD instructions that operate on vectors of data
Row-major versus column-major order
The difference between the orders lies in which elements of an array are contiguous in memory. In a row-major order, the consecutive elements of a row reside next to each other, whereas the same holds true for consecutive elements of a column in a column-major order. While the terms allude to the rows and columns of a two-dimensional array, the orders can be generalized to arrays of any dimension.
Transposition
As exchanging the indices of an array is the essence of array transposition, an array stored as row-major but read as column-major (or vice versa) will appear transposed. As actually performing this rearrangement in memory is typically an expensive operation, some systems provide options to specify individual matrices as being stored transposed.
Languages support
• Row-major: #C/C++/Objective-C (for C-style arrays), PL/I, #Pascal, Speakeasy, SAS, and Rasdaman
• Column-major: #Fortran, #MATLAB, GNU Octave, S-Plus, #R, #Julia, and Scilab.
• Neither (for less dense arrays):
• Iliffe vectors: #Java, #Scala, #Swift. #Ruby, #Perl, #PHP, #JavaScript, Visual Basic .NET
• Lists of lists: #Python, Wolfram Language of Wolfram Mathematica
• Tables of tables: #Lua
- is how multidimensional arrays are stored in a linear storage such as RAM. It is critical for:
• correctly passing arrays between programs written in different programming languages
• performance when traversing an array because modern CPUs, due to caching, process sequential data more efficiently than non-sequential data
• contiguous access makes it possible to use SIMD instructions that operate on vectors of data
Row-major versus column-major order
The difference between the orders lies in which elements of an array are contiguous in memory. In a row-major order, the consecutive elements of a row reside next to each other, whereas the same holds true for consecutive elements of a column in a column-major order. While the terms allude to the rows and columns of a two-dimensional array, the orders can be generalized to arrays of any dimension.
Transposition
As exchanging the indices of an array is the essence of array transposition, an array stored as row-major but read as column-major (or vice versa) will appear transposed. As actually performing this rearrangement in memory is typically an expensive operation, some systems provide options to specify individual matrices as being stored transposed.
Languages support
• Row-major: #C/C++/Objective-C (for C-style arrays), PL/I, #Pascal, Speakeasy, SAS, and Rasdaman
• Column-major: #Fortran, #MATLAB, GNU Octave, S-Plus, #R, #Julia, and Scilab.
• Neither (for less dense arrays):
• Iliffe vectors: #Java, #Scala, #Swift. #Ruby, #Perl, #PHP, #JavaScript, Visual Basic .NET
• Lists of lists: #Python, Wolfram Language of Wolfram Mathematica
• Tables of tables: #Lua
#Python
Iterables versus iterators
An iterable is something with an
An iterator is the result of calling
All iterators can only be iterated over once, not just those produced by generator functions.
Generators
When you call a function that contains a
Iterables versus iterators
An iterable is something with an
__iter__ method;An iterator is the result of calling
iter() on an iterable.All iterators can only be iterated over once, not just those produced by generator functions.
Generators
When you call a function that contains a
yield statement, you get a generator object, but no code runs. Then each time you extract an object from the generator, Python executes code in the function until it comes to a yield statement, then pauses and delivers the object. When you extract another object, your code will continue from where it left off i.e. it resumes just after the yield and continues until it reaches another yield (often the same one, but one iteration later). This continues until the function runs off the end, at which point the generator is deemed exhausted.#Python
The
The
filter function takes an iterable and an predicate, i.e. a function that returns a boolean, and removes items that don't match that predicate i.e. return False.#regex
Regex Cheatsheet
All the rules apply to all of the three languages: #Python, #Perl (PCRE) and #JavaScript, unless stated otherwise.
Source: debuggex.com
Basics
Quantifiers
⚠️ Default is greedy. Append ? for reluctant.
Groups
Python and PCRE:
PCRE only:
Character Classes
Assertions
Python and PCRE:
PCRE:
Flags
Python and PCRE:
JavaScript only:
Special Characters
JavaScript and PCRE:
Hexadecimal character YY (
Replacement
Python only:
JavaScript only:
Escapes
PCRE only:
POSIX Classes
PCRE only:
Regex Cheatsheet
All the rules apply to all of the three languages: #Python, #Perl (PCRE) and #JavaScript, unless stated otherwise.
Source: debuggex.com
Basics
.: Any character except newlinea: The character aab: The string aba|b: a or ba*: 0 or more a's\: Escapes a special characterQuantifiers
*: 0 or more+: 1 or more?: 0 or 1{2}: Exactly 2{2, 5}: Between 2 and 5{2,}: 2 or more(,5}: Up to 5 (Python only)⚠️ Default is greedy. Append ? for reluctant.
Groups
(...): Capturing group(?:...): Non-capturing group\Y: Match the Y'th captured groupPython and PCRE:
(?P<Y>...): Capturing group named Y(?P=Y): Match the named group Y(?#...): CommentPCRE only:
(?>...): Atomic group(?|...): Duplicate group numbers(?R): Recurse into entire pattern(?Y): Recurse into numbered group Y(?&Y): Recurse into named group Y\g{Y}: Match the named or numbered group Y\g<Y>: Recurse into named or numbered group YCharacter Classes
[ab-d]: One character of: a, b, c, d[^ab-d]: One character except: a, b, c, d[\b]: Backspace character\d: One digit\D: One non-digit\s: One whitespace\S: One non-whitespace\w: One word character\W: One non-word characterAssertions
^: Start of string$: End of string\b: Word boundary\B: Non-word boundary(?=...): Positive lookahead(?!...): Negative lookaheadPython and PCRE:
\A: Start of string, ignores m flag\Z: End of string, ignores m flag(?<=...): Positive lookbehind(?<!...): Negative lookbehind(?()|): ConditionalPCRE:
\G: Start of matchFlags
i: Ignore casem: ^ and $ match start and end of linePython and PCRE:
s: . matches newline as wellx: Allow spaces and commentsL: Locale character classesu: Unicode character classes(?iLmsux): Set flags within regexJavaScript only:
g: Global MatchSpecial Characters
\n: Newline\r: Carriage return\t: Tab\YYY: Octal character YYY\xYY: Hexadecimal character YYJavaScript and PCRE:
\0: Null character\cY: Control character YHexadecimal character YY (
\uYY for JavaScript and \x{YY} for PCRE)Replacement
Python only:
\g<0>: Insert entire match\g<Y>: Insert match Y (name or number)\Y: Insert group numbered YJavaScript only:
$$: Inserts $$&: Insert entire match$`: Insert preceding string$': Insert following string$Y: Insert Y'th captured groupEscapes
PCRE only:
\Q..\E: Remove special meaningPOSIX Classes
PCRE only:
[:alnum:]: Letters and digits[:alpha:]: Letters[:ascii:]: Ascii codes 0 - 127[:blank:]: Space or tab only[:cntrl:]: Control characters[:digit:]: Decimal digits[:graph:]: Visible characters, except space[:lower:]: Lowercase letters[:print:]: Visible characters[:punct:]: Visible punctuation characters[:space:]: Whitespace[:upper:]: Uppercase letters[:word:]: Word characters[:xdigit:]: Hexadecimal digitsQuine
- is a non-empty computer program which produces a copy of its own source code as its only output, and cannot receive any form of input, including reading a file, which means a quine is considered to be "cheating" if it looks at its own source code. Quines are possible in any Turing complete programming language, as a direct consequence of Kleene's recursion theorem.
A quine in #Python:
- is a non-empty computer program which produces a copy of its own source code as its only output, and cannot receive any form of input, including reading a file, which means a quine is considered to be "cheating" if it looks at its own source code. Quines are possible in any Turing complete programming language, as a direct consequence of Kleene's recursion theorem.
A quine in #Python:
s = 's = %r\nprint(s%%s)'
print(s%s)
#Java
The JVM
- is the detailed, formal specification for a 32-bit abstract machine. Abstract implies hardware and OS-independent and thus is interoperable across platform-dependent implementations. It operates on abstracte data types rather the native data types of any ISA:
• 32-bits types: integers, floats, and references are called primitives
• 64-bits types: long and double are supported but consume two units of storage
• smaller types: boolean, byte, short and char are extended to 32-bit ints
It loads code, verifies it, and executes it. It has instructions for:
• Load and store
• Arithmetic
• Type conversion
• Object creation and manipulation
• Operand stack management (push & pop)
• Control transfer (branching)
• Method invocation and return
• Throwing exceptions
• Monitor-based concurrency
The specification includes various dynamic features such as GC and thread management.
JDK
- is an software development kit for Java and consists of:
• a private stand-alone implementation of the JVM such as JRE's HotSpot
• the Java standard library, JCL
• an interpreter/loader (
• a compiler (
• an archiver (
• a documentation generator (Javadoc)
HotSpot
- relies on JIT; and divides the memory into generations:
• young generation: heap for short-lived objects
• old generation: heap for long-lived objects
• permanent generation: used for class definitions and associated metadata
JVM languages
Any language that can express a valid class file. The existing languages includes ports from other languages:
• JRuby (#Ruby)
• Jython (#Python)
And entirely new languages that compile to Java bytecode:
• Clojure
• Apache Groovy
• #Scala
• #Kotlin
JVM languages are compatible with each other i.e. libraries of one can be used with programs of another. A new feature will supports dynamically typed languages in the JVM, and is developed within the Da Vinci Machine project whose mission is to extend the JVM so that it supports languages other than Java.
The JVM
- is the detailed, formal specification for a 32-bit abstract machine. Abstract implies hardware and OS-independent and thus is interoperable across platform-dependent implementations. It operates on abstracte data types rather the native data types of any ISA:
• 32-bits types: integers, floats, and references are called primitives
• 64-bits types: long and double are supported but consume two units of storage
• smaller types: boolean, byte, short and char are extended to 32-bit ints
It loads code, verifies it, and executes it. It has instructions for:
• Load and store
• Arithmetic
• Type conversion
• Object creation and manipulation
• Operand stack management (push & pop)
• Control transfer (branching)
• Method invocation and return
• Throwing exceptions
• Monitor-based concurrency
The specification includes various dynamic features such as GC and thread management.
JDK
- is an software development kit for Java and consists of:
• a private stand-alone implementation of the JVM such as JRE's HotSpot
• the Java standard library, JCL
• an interpreter/loader (
java)• a compiler (
javac)• an archiver (
jar)• a documentation generator (Javadoc)
HotSpot
- relies on JIT; and divides the memory into generations:
• young generation: heap for short-lived objects
• old generation: heap for long-lived objects
• permanent generation: used for class definitions and associated metadata
JVM languages
Any language that can express a valid class file. The existing languages includes ports from other languages:
• JRuby (#Ruby)
• Jython (#Python)
And entirely new languages that compile to Java bytecode:
• Clojure
• Apache Groovy
• #Scala
• #Kotlin
JVM languages are compatible with each other i.e. libraries of one can be used with programs of another. A new feature will supports dynamically typed languages in the JVM, and is developed within the Da Vinci Machine project whose mission is to extend the JVM so that it supports languages other than Java.
#Python development tips by S. N. R.
The notebook from jupyter.org, and kivy.org for rapid GUI development
The notebook from jupyter.org, and kivy.org for rapid GUI development
OpenCL (Open Computing Language)
- is a parallelism framework for writing programs that execute across heterogeneous platforms consisting of one host CPU and any number of compute devices. These devices include GPUs, CPUs with SIMD instructions, FPGAs, Movidius Myriad 2, Adapteva epiphany and DSPs.
A compute device is broken down to several compute units, which themselves are broken down to multiple PEs (processing elements). A single function execution can run on any number of PEs in parallel. How a compute device is subdivided into compute units and PEs is up to the vendor.
It defines an API for programs running on the host to launch functions on compute devices and manage device memory. This API is defined for C and C++, as well as third-parties such as #Python, #Java, #Perl, #DotNET, etc. A more recent, higher-level model is SYCL; which is purely based on C++11. Programs in OpenCL are compiled at run-time therefore its applications are portable between various host devices.
A four-level memory hierarchy is defined for the compute device:
• global: shared by all PEs, high access latency
OpenCL C
- is the programming language used to write compute kernels. Though based on C99, it is adapted to fit the device model.
A memory buffer resides in a specific level and its pointer is annotated with a region qualifier:
• There is no
• There are no function pointers, bit fields or variable-length arrays
• Recursion is forbidden
• stdlib is replaced by a custom set of standard functions, geared toward math programming
• Scalar types such as
Features:
• Vector types available in fixed-lengths of 2, 3, 4, 8 and 16; and for various base types e.g.
• Operations for vector types
• Synchronization facilities
• Functions to work with work-items and work-groups
• More specialized types incl. 2D and 3D image types
#OpenCL #OpenCL_C
- is a parallelism framework for writing programs that execute across heterogeneous platforms consisting of one host CPU and any number of compute devices. These devices include GPUs, CPUs with SIMD instructions, FPGAs, Movidius Myriad 2, Adapteva epiphany and DSPs.
A compute device is broken down to several compute units, which themselves are broken down to multiple PEs (processing elements). A single function execution can run on any number of PEs in parallel. How a compute device is subdivided into compute units and PEs is up to the vendor.
It defines an API for programs running on the host to launch functions on compute devices and manage device memory. This API is defined for C and C++, as well as third-parties such as #Python, #Java, #Perl, #DotNET, etc. A more recent, higher-level model is SYCL; which is purely based on C++11. Programs in OpenCL are compiled at run-time therefore its applications are portable between various host devices.
A four-level memory hierarchy is defined for the compute device:
• global: shared by all PEs, high access latency
__global
• read-only: smaller, low latency, writable only by the host __constant
• local: shared by a group of PEs __local
• per-element private memory: device registers; __private
Not every device needs to implement each level of this hierarchy in hardware. Consistency between the various levels in the hierarchy is relaxed, and only enforced by explicit synchronization constructs, notably barriers. The host provides handles on device memory buffers and functions to transfer data back and forth.OpenCL C
- is the programming language used to write compute kernels. Though based on C99, it is adapted to fit the device model.
A memory buffer resides in a specific level and its pointer is annotated with a region qualifier:
__global, __local, __constant, and __private
Comparison with #C:• There is no
main; functions are marked __kernel to signal that they are entry points, and are to be called from programs running on the host• There are no function pointers, bit fields or variable-length arrays
• Recursion is forbidden
• stdlib is replaced by a custom set of standard functions, geared toward math programming
• Scalar types such as
float and double behave similarly to those of CFeatures:
• Vector types available in fixed-lengths of 2, 3, 4, 8 and 16; and for various base types e.g.
float4 (4-vector of single-precision floats)• Operations for vector types
• Synchronization facilities
• Functions to work with work-items and work-groups
• More specialized types incl. 2D and 3D image types
#OpenCL #OpenCL_C