1000 Python Examples
by: Gábor Szabó
Published: 2020
Pages: 997
Language 语言: English
Pages: PDF/EPUB
Size: 10 Mb
Book Description
First steps,What is Python?,What is needed to write a program?,The source (code) of Python,Python 2 vs. Python 3,Installation,Installation on Linux,Installation on Apple Mac OSX,Installation on MS Windows,Editors,IDEs,Documentation,Program types,Python on the command line,First script – hello world,Examples,Comments,Variables,Exercise: Hello world,What is programming? What are the programming languages,A written human language,A programming language,Words and punctuation matter!,Literals,Value Types in Python,Floating point limitation,Value Types in Numpy,Rectangular (numerical operations),Multiply string,Add numbers,Add strings,Exercise: Calculations,Solution: Calculations
Second steps
Modules
A main function
The main function – called
Indentation
Conditional main
Input – Output I/O
print in Python 2
print in Python 3
print in Python 2 as if it was Python 3
Exception: SyntaxError: Missing parentheses in call
Prompting for user input in Python 2
Prompting for user input in Python 3
Python2 input or raw_input?
Prompting both Python 2 and Python 3
Add numbers entered by: the user (oups)
Add numbers entered by: the user (fixed)
How can I check if a string can be converted to a number?
Converting string to int
Converting float to int
Conditionals: if
Conditionals: if – else
Conditionals: if – else (other example)
Conditionals: else if
Conditionals: elif
Ternary operator
Case or Switch in Python
Exercise: Rectangular
Exercise: Calculator
Exercise: Standard Input
Solution: Area of rectangular
Solution: Calculator
Command line arguments
Command line arguments – len
Command line arguments – exit
Exercise: Rectangular (argv)
Exercise: Calculator (argv)
Solution: Area of rectangular (argv)
Solution: Calculator eval
Solution: Calculator (argv)
Compilation vs. Interpretation
Is Python compiled or interpreted?
Flake8 checking
Numbers
Numbers
Operators for Numbers
Integer division and the future
Pseudo Random Number
Fixed random numbers
Rolling dice – randrange
Random choice
built-in method
Exception: TypeError: ‘module’ object is not callable
Fixing the previous code
Exception: AttributeError: module ‘random’ has no attribute
Exercise: Number guessing game – level 0
Exercise: Fruit salad
Solution: Number guessing game – level 0
Solution: Fruit salad
Boolean
if statement again
True and False
Boolean
True and False values in Python
Comparision operators
Do NOT Compare different types
Boolean operators
Boolean truth tables
Short circuit
Short circuit fixed
Incorrect use of conditions
Exercise: compare numbers
Exercise: compare strings
Solution: compare numbers
Solution: compare strings
Strings
Single quoted and double quoted strings
Long lines
Triple quoted strings (multiline)
String length (len)
String repetition and concatenation
A character in a string
String slice (instead of substr)
Change a string
How to change a string
String copy
String functions and methods (len,upper,lower)
index in string
index in string with range
rindex in string with range
find in string
Find all in the string
in string
index if in string
Encodings: ASCII,Windows-1255,Unicode
raw strings
ord
ord in a file
chr – number to character
Exercise: one string in another string
Exercise: to ASCII CLI
Exercise: from ASCII CLI
Solution: one string in another string
Solution: compare strings
Solution: to ASCII CLI
Solution: from ASCII CLI
Loops
Loops: for-in and while
for-in loop on strings
for-in loop on list
for-in loop on range
Iterable,iterator
for in loop with early end using break
for in loop skipping parts using continue
for in loop with break and continue
while loop
Infinite while loop
While with complex expression
While with break
While True
Duplicate input call
Eliminate duplicate input call
do while loop
while with many continue calls
Break out from multi-level loops
Exit vs return vs break and continue
Exercise: Print all the locations in a string
Exercise: Number guessing game
Exercise: MasterMind
Exercise: Count unique characters
Solution: Print all the locations in a string
Solution 1 for Number Guessing
Solution for Number Guessing (debug)
Solution for Number Guessing (move)
Solution for Number Guessing (multi-game)
Solution: MasterMind
Solution: Count unique characters
MasterMind to debug
PyCharm
PyCharm Intro
PyCharm Project
PyCharm Files
PyCharm – run code
PyCharm Python console at the bottom left
Refactoring example (with and without pycharm)
Formatted printing
format – sprintf
Examples using format – indexing
Examples using format with names
Format columns
Examples using format – alignment
Format – string
Format characters and types
Format floating point number
f-strings (formatted string literals)
printf using old %-syntax
Format braces,bracket,and parentheses
Examples using format with attributes of objects
raw f-strings
Lists
Anything can be a lists
Any layout
Lists
List slice with steps
Change a List
Change with steps
List assignment and list copy
join
join list of numbers
split
for loop on lists
in list
Where is the element in the list
Index improved
[].insert
[].append
[].remove
Remove element by: index [].pop
Remove first element of list
Remove several elements of list by: index
Use list as a queue
Queue using deque from collections
Fixed size queue
List as a stack
stack with deque
Exercies: Queue
Exercise: Stack
Solution: Queue with list
Solution: Queue with deque
Solution: Reverse Polish calculator (stack) with lists
Solution: Reverse Polish calculator (stack) with deque
Debugging Queue
sort
sort numbers
sort mixed
key sort
Sort tuples
sort with sorted
sort vs. sorted
key sort with sorted
Sorting characters of a string
range
Looping over index
Enumerate lists
List operators
List of lists
List assignment
List documentation
tuple
Exercise: color selector menu
Exercise: count digits
Exercise: Create list
Exercise: Count words
Exercise: Check if number is prime
Exercise: DNA sequencing
Solution: menu
Solution: count digits
Solution: Create list
Solution: Count words
Solution: Check if number is prime
Solution: DNA sequencing
Solution: DNA sequencing with filter
Solution: DNA sequencing with filter and lambda
[].extend
append vs. extend
split and extend
Files
Open and read file
Filename on the command line
Filehandle with and without
Filehandle with return
Read file remove newlines
Read all the lines into a list
Read all the characters into a string (slurp)
Not existing file
Open file exception handling
Open many files – exception handling
Writing to file
Append to file
Binary mode
Does file exist? Is it a file?
Exercise: count numbers
Exercise: strip newlines
Exercise: color selector
Exercise: ROT13
Exercise: Combine lists
Solution: count numbers
Solution: strip newlines
Solution: color selector
Solution: Combine lists
Read text file
Open and read file
Direct access of a line in a file
Example
Dictionary (hash)
What is a dictionary
When to use dictionaries
Dictionary
keys
Loop over keys
Loop using items
values
Not existing key
Get key
Does the key exist?
Does the value exist?
Delete key
List of dictionaries
Shared dictionary
immutable collection: tuple as dictionary key
immutable numbers: numbers as dictionary key
Sort dictionary by: value
Sort dictionary keys by: value
Insertion Order is kept
Change order of keys in dictionary – OrderedDict
Set order of keys in dictionary – OrderedDict
Exercise: count characters
Exercise: count words
Exercise: count words from a file
Exercise: Apache log
Exercise: Combine lists again
Exercise: counting DNA bases
Exercise: Count Amino Acids
Exercise: List of dictionaries
Exercise: Dictinoary of dictionaries
Solution: count characters
Solution: count characters with default dict
Solution: count words
Solution: count words in file
Solution: Apache log
Solution: Combine lists again
Solution: counting DNA bases
Solution: Count Amino Acids
Loop over dictionary keys
Do not change dictionary in loop
Default Dict
Sets
sets
set operations
set intersection
set subset
set symmetric difference
set union
set relative complement
set examples
defining an empty set
Adding an element to a set (add)
Merging one set into another set (update)
Functions (subroutines)
Defining simple function
Defining a function
Parameters can be named
Mixing positional and named parameters
Default values
Several defaults,using names
Arbitrary number of arguments *
Fixed parmeters before the others
Arbitrary key-value pairs in parameters **
Extra key-value pairs in parameters
Every parameter option
Duplicate declaration of functions (multiple signatures)
Recursive factorial
Recursive Fibonacci
Non-recursive Fibonacci
Unbound recursion
Variable assignment and change – Immutable
Variable assignment and change – Mutable
Parameter passing of functions
Passing references
Function documentation
Sum ARGV
Copy-paste code
Copy-paste code fixed
Copy-paste code further improvement
Palindrome
Exercise: statistics
Exercise: recursive
Exercise: Tower of Hanoi
Exercise: Merge and Bubble sort
Solution: statistics
Solution: recursive
Solution: Tower of Hanoi
Solution: Merge and Bubble sort
Modules
Before modules
Create modules
path to load modules from – The module search path
sys.path – the module search path
Flat project directory structure
Absolute path
Relative path
Python modules are compiled
How “import” and “from” work?
Runtime loading of modules
Conditional loading of modules
Duplicate importing of functions
Script or library
Script or library – import
Script or library – from import
assert to verify values
mycalc as a self testing module
doctest
Scope of import
Export import
Export import with all
import module
Execute at import time
Import multiple times
Exercise: Number guessing
Exercies: Scripts and modules
Exercise: Module my_sum
Exercise: Convert your script to module
Exercise: Add doctests to your own code
Solution: Module my_sum
Regular Expressions
What are Regular Expressions (aka. Regexes)?
What are Regular Expressions good for?
Examples
Where can I use it ?
grep
Regexes first match
Match numbers
Capture
Capture more
Capture even more
findall
findall with capture
findall with capture more than one
Any Character
Match dot
Character classes
Common characer classes
Negated character class
Optional character
Regex 0 or more quantifier
Quantifiers
Quantifiers limit
Quantifiers on character classes
Greedy quantifiers
Minimal quantifiers
Anchors
Anchors on both end
Match ISBN numbers
Matching a section
Matching a section – minimal
Matching a section negated character class
DOTALL S (single line)
MULTILINE M
Two regex with logical or
Alternatives
Grouping and Alternatives
Internal variables
More internal variables
Regex DNA
Regex IGNORECASE
Regex VERBOSE X
Substitution
findall capture
Fixing dates
Duplicate numbers
Remove spaces
Replace string in assembly code
Full example of previous
Split with regex
Exercises: Regexes part 1
Exercise: Regexes part 2
Exercise: Sort SNMP numbers
Exercise: parse hours log file and give report
Exercise: Parse ini file
Exercise: Replace Python
Exercise: Extract phone numbers
Solution: Sort SNMP numbers
Solution: parse hours log file and give report
Solution: Processing INI file manually
Solution: Processing config file
Solution: Extract phone numbers
Regular Expressions Cheat sheet
Fix bad JSON
Fix very bad JSON
Raw string or escape
Remove spaces regex
Regex Unicode
Anchors Other example
Python standard modules
Some Standard modules
sys
Writing to standard error (stderr)
Current directory (getcwd,pwd,chdir)
OS dir (mkdir,makedirs,remove,rmdir)
python which OS are we running on (os,platform)
Get process ID
OS path
Traverse directory tree – list directories recursively
os.path.join
Directory listing
expanduser – handle tilde ~
Listing specific files using glob
External command with system
subprocess
subprocess in the background
Accessing the system environment variables from Python
Set env and run command
shutil
time
sleep in Python
timer
Current date and time datetime now
Converting string to datetime
datetime arithmeticis
Rounding datetime object to nearest second
Signals and Python
Sending Signal
Catching Signal
Catching Ctrl-C on Unix
Catching Ctrl-C on Unix confirm
Alarm signal and timeouts
deep copy list
deep copy dictionary
Exercise: Catching Ctrl-C on Unix 2nd time
Exercise: Signals
Ctrl-z
JSON
JSON – JavaScript Object Notation
dumps
loads
dump
load
Round trip
Pretty print JSON
Sort keys in JSON
Set order of keys in JSON – OrderedDict
Exercise: Counter in JSON
Exercise: Phone book
Exercise: Processes
Solution: Counter in JSON
Solution: Phone book
Command line arguments with argparse
Modules to handle the command line
argparse
Basic usage of argparse
Positional argument
Many positional argument
Convert to integers
Convert to integer
Named arguments
Boolean Flags
Short names
Exercise: Command line parameters
Exercise: argparse positional and named
Exception handling
Hierarchy of calls
Handling errors as return values
Handling errors as exceptions
A simple exception
Working on a list
Catch ZeroDivisionError exception
Module to open files and calculate something
File for exception handling example
Open files – exception
Handle divide by: zero exception
Handle files – exception
Catch all the exceptions and show their type
List exception types
Exceptions
How to raise an exception
Stack trace
Exercies: Exception int conversion
Exercies: Raise Exception
Solution: Exception int conversion (specific)
Solution: Exception int conversion (all other)
Solution: Raise Exception
Classes – OOP – Object Oriented Programming
Why Object Oriented Programming?
Generic Object Oriented Programming terms
OOP in Python
OOP in Python (numbers,strings,lists)
OOP in Python (argparse)
Create a class
Import module containing class
Import class from module
Initialize a class – constructor,attributes
Attributes are not special
Create Point class
Initialize a class – constructor,attributes
Methods
Stringify class
Inheritance
Inheritance – another level
Modes of method inheritance
Modes of method inheritance – implicit
Modes of method inheritance – override
Modes of method inheritance – extend
Modes of method inheritance – delegate – provide
Composition – Line
Some comments
Class in function
Serialization of instances with pickle
Quick Class definition and usage
Exercise: Add move_rad to based on radians
Exercise: Improve previous examples
Exercise: Polygon
Exercise: Number
Exercise: Library
Exercise: Bookexchange
Exercise: Represent turtle graphics
Solution – Polygon
PyPi – Python Package Index
What is PyPi?
Easy Install
pip
Upgrade pip
PYTHONPATH
Virtualenv
Virtualenv for Python 3
SQLite Database Access
SQLite
Connecting to SQLite database
Create TABLE in SQLite
INSERT data into SQLite database
SELECT data from SQLite database
A counter
MySQL
Install MySQL support
Create database user (manually)
Create database (manually)
Create table (manually)
Connect to MySQL
Connect to MySQL and Handle exception
Select data
Select more data
Select all data fetchall
Select some data fetchmany
Select some data WHERE clause
Select into dictionaries
Insert data
Update data
Delete data
Exercise MySQL
Exercise: MySQL Connection
Solution: MySQL Connection
PostgreSQL
PostgreSQL install
Python and Postgresql
PostgreSQL connect
INSERT
INSERT (from command line)
SELECT
DELETE
SQLAlchemy
SQLAlchemy hierarchy
SQLAlchemy engine
SQLAlchemy autocommit
SQLAlchemy engine CREATE TABLE
SQLAlchemy engine INSERT
SQLAlchemy engine SELECT
SQLAlchemy engine SELECT all
SQLAlchemy engine SELECT fetchall
SQLAlchemy engine SELECT aggregate
SQLAlchemy engine SELECT IN
SQLAlchemy engine SELECT IN with placeholders
SQLAlchemy engine connection
SQLAlchemy engine transaction
SQLAlchemy engine using context managers
Exercise: Create table
SQLAlchemy Metada
SQLAlchemy types
SQLAlchemy ORM – Object Relational Mapping
SQLAlchemy ORM create
SQLAlchemy ORM schema
SQLAlchemy ORM reflection
SQLAlchemy ORM INSERT after automap
SQLAlchemy ORM INSERT
SQLAlchemy ORM SELECT
SQLAlchemy ORM SELECT cross tables
SQLAlchemy ORM SELECT and INSERT
SQLAlchemy ORM UPDATE
SQLAlchemy ORM logging
Solution: Create table
Exercise: Inspector
SQLAlchemy CREATE and DROP
SQLAlchemy Notes
SQLAlchemy Meta SQLite CREATE
SQLAlchemy Meta Reflection
SQLAlchemy Meta INSERT
SQLAlchemy Meta SELECT
NoSQL
Types of NoSQL databases
MongoDB
MongoDB CRUD
Install MongoDB support
Python MongoDB insert
MongoDB CLI
Python MongoDB find
Python MongoDB find refine
Python MongoDB update
Python MongoDB remove (delete)
Redis
Redis CLI
Redis list keys
Redis set get
Redis incr
Redis incrby:
Redis setex
Web client
urllib the web client
urllib2 the web client
httpbin.org
requests get
Download image using requests
Download image as a stream using requests
Download zip file
Extract zip file
Interactive Requests
requests get JSON
requests get JSON UserAgent
requests get JSON UserAgent
requests get header
requests change header
requests post
Tweet
API config file
bit.ly
Exercise: Combine web server and client
Python Web server
Hello world web
Dump web environment info
Web echo
Web form
Resources
Python Flask
Python Flask intro
Python Flask installation
Flask: Hello World
Flask hello world + test
Flask generated page – time
Flask: Echo GET
Flask: Echo POST
Flask: templates
Flask: templates
Flask: templates with parameters
Flask: runner
Exercise: Flask calculator
Static files
Flask Logging
Flask: Counter
Color selector without session
Session management
Flask custom 404 page
Flask Error page
Flask URL routing
Flask Path params
Flask Path params (int)
Flask Path params add (int)
Flask Path params add (path)
Jinja loop,conditional,include
Exercise: Flask persistent
Exercise: Flask persistent
Flask Exercises
Flask login
Flask JSON API
Flask and AJAX
Flask and AJAX
passlib
Flask Testing
Flask Deploy app
Flask Simple Authentication + test
Flask REST API
Flask REST API – Echo
Flask REST API – parameters in path
Flask REST API – parameter parsing
Flask REST API – parameter parsing – required
Networking
Secure shell
ssh
ssh from Windows
Parallel ssh
telnet
prompt for password
Python nmap
ftp
Interactive shell
The Python interactive shell
REPL – Read Evaluate Print Loop
Using Modules
Getting help
Exercise: Interactive shell
Testing Demo
How do you test your code?
What is testing?
What is testing really?
Testing demo – AUT – Application Under Test
Testing demo – use the module
Testing demo: doctets
Testing demo: Unittest success
Testing demo: Unittest failure
Testing demo: pytest using classes
Testing demo: pytest without classes
Testing demo: pytest run doctests
Testing demo: pytest run unittest
Exercise: Testing demo
Solution: Testing demo
Types in Python
mypy
Types of variables
Types of function parameters
Types used properly
TODO: mypy
Testing Intro
The software testing equasion
The software testing equasion (fixed)
The pieces of your software?
Manual testing
What to tests?
Continuous Integration
Functional programming
Functional programming
Iterators (Iterables)
range
range with list
range vs. list size
for loop with transformation
map
map delaying function call
map on many values
map with list
double with lambda
What is lambda in Python?
lambda returning tuple
map returning tuples
lambda with two parameters
map for more than one iterable
map on uneven lists
replace None (for Python 2)
map on uneven lists – fixed (for Python 2)
map mixed iterators
map fetch value from dict
Exercise: string to length
Exercise: row to length
Exercise: compare rows
Solution: string to length
Solution: row to length
Solution: compare rows
filter
filter with lambda
filter – map example
filter – map in one expression
Get indexes of values
reduce
reduce with default
zip
Creating dictionary from two lists using zip
all,any
Compare elements of list with scalar
List comprehension – double
List comprehension – simple expression
List generator
List comprehension
Dict comprehension
Lookup table with lambda
Read lines without newlines
Read key-value pairs
Create index-to-value mapping in a dictionary based on a list of values
Exercise: min,max,factorial
Exercise: Prime numbers
Exercise: Many validator functions
Exercise: Calculator using lookup table
Exercise: parse file
Solution: min,max,factorial
Solution: Prime numbers
Solution: Many validator functions
Solution: Calculator using lookup table
map with condtion
map with lambda
map with lambda with condition
List comprehension – complex
Iterators – with and without Itertools
Advantages of iterators and generators
The Fibonacci research institute
Fibonacci plain
Fibonacci copy-paste
Iterators Glossary
What are iterators and iterables?
A file-handle is an iterator
range is iterable but it is not an iterator
Iterator: a counter
Using iterator
Iterator without temporary variable
The type of the iterator
Using iterator with next
Mixing for and next
Iterable which is not an iterator
Iterator returning multiple values
Range-like iterator
Unbound or infinite iterator
Unbound iterator Fibonacci
Operations on Unbound iterator
itertools
itertools – count
itertools – cycle
Exercise: iterators – reimplement the range function
Exercise: iterators – cycle
Exercise: iterators – alter
Exercise: iterators – limit Fibonacci
Exercise: iterators – Fibonacci less memory
Exercise: read char
Exercise: read section
Exercise: collect packets
Exercise: compare files
Solution: iterators – limit Fibonacci
Solution: iterators – Fibonacci less memory
Solution: read section
Solution: compare files
Solution: collect packets
Generators and Generator Expressions
Generators Glossary
Iterators vs Generators
List comprehension and Generator Expression
List comprehension vs Generator Expression – less memory
List comprehension vs Generator Expression – lazy evaluation
Generator: function with yield – call next
Generators – call next
Generator with yield
Generators – fixed counter
Generators – counter
Generators – counter with parameter
Generators – my_range
Fibonacci – generator
Infinite series
Integers
Integers + 3
Integers + Integers
Filtered Fibonacci
The series.py
generator – unbound count (with yield)
iterator – cycle
Exercise: Alternator
Exercise: Prime number generator
Exercise: generator
Exercise: Tower of Hanoi
Exercise: Binary file reader
Exercise: File reader with records
Logging
Simple logging
Simple logging – set level
Simple logging to a file
Simple logging format
Simple logging change date format
getLogger
Time-based logrotation
Size-based logrotation
Closures
Counter local – not working
Counter with global
Create incrementors
Create internal function
Create function by: a function
Create function with parameters
Counter closure
Make incrementor with def (closure)
Make incrementor with lambda
Exercise: closure bank
Solution: closure bank
Solution: counter with parameter
Decorators
Function assignment
Function inside other function
Decorator
Use cases for decorators in Python
A recursive Fibonacci
trace fibo
tron decorator
Decorate with direct call
Decorate with parameter
Decorator accepting parameter
Decorate function with any signature
Decorate function with any signature – implementation
Exercise: Logger decorator
Exercise: memoize decorator
Solution: Logger decorator
Solution: Logger decorator (testing)
Solution memoize decorator
Context managers (with statement)
Why use context managers?
Context Manager examples
cd in a function
open in function
open in for loop
open in function using with
Plain context manager
Param context manager
Context manager that returns a value
Use my tempdir – return
Use my tempdir – exception
cwd context manager
tempdir context manager
Context manager with class
Context managers with class
Context manager: with for file
With – context managers
Exercise: Context manager
Exercise: Tempdir on Windows
Solution: Context manager
Advanced lists
Change list while looping: endless list
Change list while looping
Copy list before iteration
for with flag
for else
enumerate
do while
list slice is copy
Advanced Exception handling
Exceptions else
Exceptions finally
Exit and finally
Catching exceptions
Home made exception
Home made exception with attributes
Home made exception hierarcy
Home made exception hierarcy – 1
Home made exception hierarcy – 2
Home made exception hierarcy – 3
Exercise: spacefight with exceptions
Exercies: Raise My Exception
Solution: spacefight with exceptions
Solution: Raise My Exception
Exception finally return
Warnings
Warnings
CSV
Reading CSV the naive way
CSV with quotes and newlines
Reading a CSV file
CSV dialects
CSV to dictionary
Exercise: CSV
Solution: CSV
Excel
Spreadsheets
Python Excel
Create an Excel file from scratch
Worksheets in Excel
Add expressions to Excel
Format field
Number series and chart
Read Excel file
Update Excel file
Exercise: Excel
XML
XML Data
Expat – Callbacks
XML DOM – Document Object Model
XML SAX – Simple API for XML
SAX collect
XML elementtree
SciPy – for Scientific Computing in Python
Data Science tools in Python
Data Analysis resources
Python and Biology
Biopython
Biopython background
Bio python sequences
Download data
Read FASTA,GenBank files
Search nucleotids
Download nucleotids
Exercise: Nucleotid
Biology background
Chemistry
Chemistry links
Bond length
Covalent radius
Python energy landscape explorer
Other chemistry links
numpy
What is NumPy
Numpy – vector
NumPy 2D arrays
Numpy – set type
NumPy arrays: ones and zeros
Numpy: eye
NumPy array random
NumPy Random integers
NumPy array type change by: division (int to float)
Numpy: Array methods: transpose
Numpy: reference,not copy
Numpy: copy array
Numpy: Elementwise Operations on Arrays
Numpy: multiply,matmul,dot for vectors
Numpy: multiply,matmul,dot for vector and matrix
Numpy: multiply,matmul,dot for matrices
Numpy: casting – converting from strings to integer.
Numpy: indexing 1d array
Numpy: slice is a reference
Numpy: slice – copy
Numpy: abs value on a Numpy array
Numpy: Logical not on a Numpy array
Numpy: Vectorize a function
Numpy: Vectorize len
Numpy: Vectorize lambda
Numpy: Filtering array
Numpy: Filter matrix values
Numpy: Filter matrix rows
Numpy: Stat
Numpy: Serialization
Numpy: Load from Matlab file
Numpy: Save as Matlab file
Numpy: Horizontal stack vectors (hstack)
Numpy: Append or vertically stack vectors and matrices (vstack)
Numpy uint8
Numpy int8
Pandas
Pandas
Planets
Pandas Planets – Dataframes
Pandas Stocks
Pandas Stocks
Merge Dataframes
Analyze Alerts
Analyze IFMetrics
Create Excel file for experiment with random data
Calculate Genome metrics
Calculate Genome metrics – add columns
Calculate Genome metrics – vectorized
Calculate Genome metrics – vectorized numpy
Genes using Jupyter
Combine columns
Pandas more
Pandas Series
Pandas Series with names
Matplotlib
About Matplotlib
Matplotlib Line
Matplotlib Line with dates
Matplotlib Simple Pie
Matplotlib Simple Pie with params
Matplotlib Pie
Matplotlib Pie 2
Plot,scatter,histogram
Seaborn
Searborn use examples
Seaborn tip
Seaborn Anscombes Quartet
Jupyter notebooks
Jupyter on Windows
Jupyter on Linux and OSX
Jupyter add
Planets
Jupyter notebook Planets
Jupyter StackOverflow
Jupyter StackOverflow – selected columns
Jupyter processing chunks
Jupyter StackOverflow – selected rows
Jupyter StackOverflow – biggest countries (in terms of number of responses)
Jupyter StackOverflow – historgram
Jupyter StackOverflow – filter by: country
Jupyter StackOverflow – OpenSourcer
Jupyter StackOverflow – cross tabulation
Jupyter StackOverflow – salaries
Jupyter StackOverflow – replace values
Jupyter StackOverflow – selected rows
Jupyter notebook Intellisense (TAB completition)
Jupyter examples
IPy Widgets
Testing
Traditional Organizations
Quality Assurance
Web age Organizations
TDD vs Testing as an Afterthought
Why test?
Testing Modes
Testing Applications
Testing What to test?
Testing in Python
Testing Environment
Testing Setup – Fixture
Testing Resources
Testing with unittest
Use a module
Test a module
The tested module
Testing – skeleton
Testing
Test examples
Testing with PyTest
Pytest features
Pytest setup
Testing with Pytest
Testing functions
Testing class and methods
Pytest – execute
Pytest – execute
Pytest simple module to be tested
Pytest simple tests – success
Pytest simple tests – success output
Pytest simple tests – failure
Pytest simple tests – failure output
Exercise: test math functions
Exercise: test this app
Exercise: test the csv module
Solution: Pytest test math functions
Solution: Pytest test this app
Solution: test the csv module
PyTest bank deposit
PyTest expected exceptions (bank deposit)
PyTest expected exceptions (bank deposit) – no exception happens
PyTest expected exceptions (bank deposit) – different exception is raised
PyTest expected exceptions
PyTest expected exceptions output
PyTest expected exceptions (text changed)
PyTest expected exceptions (text changed) output
PyTest expected exceptions (other exception)
PyTest expected exceptions (other exception) output
PyTest expected exceptions (no exception)
PyTest expected exceptions (no exception) output
PyTest: Multiple Failures
PyTest: Multiple Failures output
PyTest Selective running of test functions
PyTest: stop on first failure
Pytest: expect a test to fail (xfail or TODO tests)
Pytest: expect a test to fail (xfail or TODO tests)
PyTest: show xfailed tests with -rx
Pytest: skipping tests
Pytest: show skipped tests woth -rs
Pytest: show extra test summmary info with -r
Pytest: skipping tests output in verbose mode
Pytest verbose mode
Pytest quiet mode
PyTest print STDOUT and STDERR using -s
PyTest failure reports
PyTest compare numbers
PyTest compare numbers relatively
PyTest compare strings
PyTest compare long strings
PyTest is one string in another strings
PyTest test any expression
PyTest element in list
PyTest compare lists
PyTest compare short lists
PyTest compare short lists – verbose output
PyTest compare dictionaries
PyTest compare dictionaries output
PyTest Fixtures
PyTest Fixture setup and teardown
PyTest Fixture setup and teardown output
PyTest: Class setup and teardown
PyTest: Class setup and teardown output
Pytest Dependency injection
Pytest fixture – tmpdir
Pytest capture STDOUT and STDERR with capsys
Pytest Fixture – home made fixtures
More fixtures
Pytest: Mocking – why?
Pytest: Mocking – what?
Pytest: One dimensional spacefight
Pytest: Mocking input and output
Pytest: Mocking random
Pytest: Flask echo
Pytest: testing Flask echo
PyTest: Run tests in parallel with xdist
PyTest: Order of tests
PyTest: Randomize Order of tests
PyTest: Force default order
PyTest: no random order
Anagram on the command line
PyTest testing CLI
PyTest test discovery
PyTest test discovery – ignore some tests
PyTest select tests by: name
PyTest select tests by: marker
PyTest: Test Coverage
Exercise: module
Exercise: Open Source
Pytest resources
Pytest and tempdir
PyTest compare short lists – output
PyTest with parameter
PyTest with parameters
Pytest reporting in JUnit XML format
No test selected
Advancted functions
Variable scopes
Name resolution order (LEGB)
Scoping: global seen from fuction
Assignment creates local scope
Local scope gone wrong
Changing global variable from a function
Global variables mutable in functions
Scoping issues
sub in sub
Scoping sub in sub (enclosing scope)
Function objects
Functions are created at run time
Mutable default
Use None as default parameter
Inner function created every time the outer function runs
Static variable
Static variable in generated function
Inspect
Variable number of function arguments
Python function arguments – a reminder
Functions with unknown number of argumerns
Variable length argument list with * and **
Passing arguments as they were received (but incorrectly)
Unpacking args before passing them on
Exercise: implement the my_sum function
Solution: implement the my_sum function
Exercise: implement the reduce function
Soluton: implement the reduce function
Exercise: sort pairs
Solution: sort pairs
Python Packages
Why Create package
Create package
Internal usage
use module in package – relative path
use package (does not work)
package importing (and exporting) module
use package (module) with import
use package with import
Creating an installable Python package
Create tar.gz file
Install Package
Dependencies
Add README file
Add README file (setup.py)
Include executables
Add tests
Add tests calc
Add tests all
setup.py
Run tests and create package
Packaging applications (creating executable binaries)
Using PyInstaller
Other PyInstaller examples
Other
Py2app for Mac
Exercise: package
Exercise: create executable
Ctypes
ctypes – hello
concat
links
Advanced OOP
Class count instances
Class Attributes
Class Attributes in Instances
Attributes with method access
Instance Attribute
Methods are class attributes
Monkey patching
Classes: instance method
Class methods and class attributes
Classes: constructor
Class methods – alternative constructor
Abstract Base Class
Abstract Base Class with abc
ABC working example
ABC – cannot instantiate the base-class
ABC – must implement methods
Use Python @propery to fix bad interface (the bad interface)
Use Python @propery to fix bad interface (first attempt)
Use Python @propery to fix bad API
Use Python @propery decorator to fix bad API
Use Python @propery for value validation
class and static methods
Destructor: del
Destructor delayed
Destructor delayed for both
Opearator overloading
Operator overloading methods
Exercise: rectangular
Exercise: SNMP numbers
Exercise: Implement a Gene inheritance model combining DNA
Exercise: imaginary numbers – complex numbers
Solution: Rectangular
Solution: Implement a Gene inheritance model combining DNA
Instance counter
2to3
Convertig from Python 2 to Python 3
division
print in Python 2
print in Python 3
input and raw_input
Code that works on both 2 and 3
Compare different types
Octal numbers
2to3 Resources
Design Patterns
What are Design Patterns?
Don’t replace built-in objects
Facade – simple interface to complex system
Monkey Patching
Creation DPs “Just One”
Singleton
Monostate (Borg)
Dispatch table
Parallel
Types of Problems
Types of solutions
How many parallels to use?
Dividing jobs
Performance Monitoring
Threads
Python Threading docs
Threaded counters
Simple threaded counters
Simple threaded counters (parameterized)
Pass parameters to threads – Counter with attributes
Create a central counter
Lock – acquire – release
Counter – plain
GIL – Global Interpreter Lock
Thread load
Exercise: thread files
Exercise: thread URL requests.
Exercise: thread queue
Solution: thread queue
Solution: thread URL requests.
Forking
Fork
Forking
Fork skeleton
Fork with load
Fork load results
Marshalling / Serialization
Fork with random
Exercise: fork return data
Solution: fork return data
Asyncronus programming with AsyncIO
Sync chores
Async chores
Explanation
Coroutines
More about asyncio
Async files
Asynchronus programming with Twisted
About Twisted
Echo
Echo with log
Simple web client
Web client
Multiprocess
Multiprocess CPU count
Multiprocess Process
Multiprocess N files: Pool
Multiprocess load
Multiprocess: Pool
Multiprocess load async
Multiprocess and logging
Exercise: Process N files in parallel
Exercise: Process N Excel files in parallel
Exercise: Fetch URLs in parallel
Exercise: Fetch URLs from one site.
Solution: Fetch URLs in parallel
Multitasking
What is Multitasking?
Multitasking example
Multitasking example with wait
Multitaksing – second loop waits for first one
Multitasking counter
Multitasking counter with thread locking
Improving Performance – Optimizing code
Problems
Optimization strategy
Locate the source of the problem
Optimizing tactics
DSU: Decorate Sort Undecorate
Profile code
Slow example
profile slow code
cProfile slow code
Benchmarking
Benchmarking subs
Levenshtein distance
Generate words
Levenshtein – pylev
Levenshtein – edittidtance
Editdistance benchmark
A Tool to Generate text files
Count characters
Memory leak
Garbage collection
Weak reference
Exercise: benchmark list-comprehension,map,for
Exercise: Benchmark Levenshtein
Exercise: sort files
Exercise: compare split words:
Exercise: count words
GUI with Python/Tk
Sample Tk app
GUI Toolkits
Installation
Python Tk Documentation
Python Tk Button
Python Tk Button with action
Python Tk Label
Python Tk Label – font size and color
Python Tk Keybinding
Python Tk Entry (one-line text entry)
Python Tk Entry for passwords and other secrets (hidden text)
Python Tk Checkbox
Python Tk Radiobutton
Python Tk Listbox
Python Tk Listbox Multiple
Python Tk Menubar
Python Tk Text
Python Tk Dialogs
Python Tk Filedialog
Python Tk messagebox
Python Tk Combobox
Python Tk OptionMenu
Python Tk Scale
Python Tk Progressbar
Python Tk Frame
Not so Simple Tk app with class
Tk: Hello World
Tk: Quit button
Tk: File selector
Tk: Checkbox
Tk: Runner
Tk: Runner with threads
Getting started with Tk
Exercise: Tk – Calculator one line
Exercise: Tk Shopping list
Exercise: Tk TODO list
Exercise: Tk Notepad
Exercise: Tk Copy files
Exercise: Tk
Solution: Tk – Calculator one line
Solution: Tk
Solution: Tk Notepad
Simple file dialog
Python Pitfalls
Reuse of existing module name
Use the same name more than once
Compare string and number
Compare different types
Sort mixed data
Linters
Static Code Analyzis – Linters
PEP8
F811 – redefinition of unused
Warn when Redefining functions
Python .NET
IronPython
Use .NET libraries from Python
Python and .NET console
Python and .NET examples
Exercise Python and .NET
Python and Java
Jython
Calling Java from Python
Jython – Python running on the JVM
Jython Installation
Jython Installation
Jython load Java class
Jython load Java class in code
Jython test Java class
PIL – Pillow
Install Pillow
Create First Image
Write Text on Image
Select font for Text on Image
Font directories
Get size of an Image
Get size of text
Resize an existing Image
Crop an existing Image
Combine two images
Rotated text
Rotated text in top-right corner
Embed image (put one image on another one)
Draw a triangle
Draw a triangle and write text in it
Draw a triangle and write rotated text in it
Draw a rectangular
Draw a rectangle
Draw circle
Draw heart
Rectangle with rounded corners
TODO
FAQ
How not to name example scirpts?
Platform independent code
How to profile a python code to find causes of slowness?
pdb = Python Debugger
Avoid Redefining functions
Appendix
print_function
Dividers (no break or continue)
Lambdas
Abstract Class
Remove file
Modules: more
import hooks
Python resources
Progress bar
from future
Variable scope
scope
type
Look deeper in a list
Exercise: iterators – count
Simple function (before generators)
Other slides
Other slides
Atom for Python
IDLE – Integrated DeveLopment Environment
sh-bang – executable on Linux/Apple
Strings as Comments
pydoc
How can I check if a string can be converted to a number?
Spyder Intro
Interactive Debugging
Parameter passing
Command line arguments and main
Infinite loop
break
continue
While with many conditions
while loop with many conditions
Format with conversion (stringifiation with str or repr)
Name of the current function in Python
Name of the caller function in Python
Stack trace in Python using inspect
Module Fibonacci
PyTest – assertion
PyTest – failure
PyTest – list
SAX with coroutine
Getting the class name of an object
Inheritance – super
Inheritance – super – other class
iterator – pairwise
iterator – grouped
itertools – groupby:
Circular references
Context managers: with (file) experiments
itertools – izip
mixing iterators
mixing iterators
itertools – pairwise
itertools – grouped
range vs xrange in Python
profile (with hotshot) slow code
Abstract Base Class without abc
Abstract Base Class with abc Python 2 ?
Abstract Base Class with metaclass
Create class with metaclass
Python Descriptors
alter iterator
Create a counter queue
A Queue of tasks
Filtered Fibonacci with ifilter
Python from .NET
