Python 2.1

The Python 2.1 Bible fills a critical void in the Python reference market. Although it includes a complete Python language reference section, it is still geared towards those of you who already have some programming experience. This book explains each piece of technology in depth and shows through clear examples why each feature is useful. This is the manual you've been waiting for -- the one that covers all major Python components without glossing over how the various pieces fit together.

Contents

Preface

Acknowledgments

Part I: The Python Language 1

Chapter 1: Python in an Hour

Jumping In: Starting the Python Interpreter

Experimenting with Variables and Expressions

Pocket calculator

Variables

Defining a Function

Running a Python Program

Looping and Control

Integer division

Looping

Branching with if-statements

Breaking and continuing

Lists and Tuples

Tuples

Slicing and dicing

Dictionaries

Reading and Writing Files

Sample Program: Word Frequencies

Loading and Using Modules

Creating a Class

Some quick object jargon

Object orientation, Python style

Keep off the grass -- Accessing class members

Example: the point class

Recommended Reading

Chapter 2: Identifiers, Variables, and Numeric Types

Identifiers and Operators

Reserved words

Operators

Numeric Types

Integers

Long integers

Floating point numbers

Imaginary numbers

Manipulating numeric types

Assigning Values to Variables

Simple assignment statements

Multiple assignment

Augmented assignment

Chapter 3: Expressions and Strings

Expressions

Comparing numeric types

Compound expressions

Complex expressions

Operator precedence

Strings

String literals

Manipulating strings

Comparing strings

Unicode string literals

Converting Between Simple Types

Converting to numerical types

Converting to strings

Chapter 4: Advanced Data Types

Grouping Data with Sequences

Creating lists

Creating tuples

Working with Sequences

Joining and repeating with arithmetic operators

Comparing and membership testing

Accessing parts of sequences

Iterating with for...in

Using sequence utility functions

Using Additional List Object Features

Additional operations

List object methods

Mapping Information with Dictionaries

Creating and adding to dictionaries

Accessing and updating dictionary mappings

Additional dictionary operations

Understanding References

Object identity

Counting references

Copying Complex Objects

Shallow copies

Deep copies

Identifying Data Types

Working with Array Objects

Creating arrays

Converting between types

Array methods and operations

Chapter 5: Control Flow

Making Decisions with If-Statements

Using For-Loops

Anatomy of a for-loop

Looping example: encoding strings

Ranges and xranges

Breaking, continuing, and else-clauses

Changing horses in midstream

Using While-Loops

Throwing and Catching Exceptions

Passing the buck: propagating exceptions

Handling an exception

More on exceptions

Defining and raising exceptions

Cleaning up with finally

Debugging with Assertions

Assertions in Python

Toggling assertions

Example: Game of Life

Chapter 6: Program Organization

Defining Functions

Pass by object reference

All about parameters

Arbitrary arguments

Apply: passing arguments from a tuple

A bit of functional programming

Grouping Code with Modules

Taking inventory of a module

Importing Modules

What else happens upon import?

Reimporting modules

Exotic imports

Locating Modules

Python path

Compiled files

Understanding Scope Rules

Is it local or global?

Listing namespace contents

Grouping Modules into Packages

Compiling and Running Programmatically

Chapter 7: Object-Oriented Python

Overview of Object-Oriented Python

Creating Classes and Instance Objects

Creating instance objects

More on accessing attributes

Deriving New Classes from Other Classes

Multiple inheritance

Creating a custom list class

Creating a custom string class

Creating a custom dictionary class

Hiding Private Data

Identifying Class Membership

Overloading Standard Behaviors

Overloading basic functionality

Overloading numeric operators

Overloading sequence and dictionary operators

Overloading bitwise operators

Overloading type conversions

Using Weak References

Creating weak references

Creating proxy objects

Chapter 8: Input and Output

Printing to the Screen

Accessing Keyboard Input

raw_input

input

Opening, Closing, and Positioning Files

open

File object information

close

File position

Writing Files

Reading Files

Accessing Standard I/O

Using Filelike Objects

Chapter 9: Processing Strings and Regular Expressions

Using String Objects

String formatting methods

String case-changing methods

String format tests (the is-methods)

String searching methods

String manipulation methods

Using the String Module

Character categories

Miscellaneous functions

Defining Regular Expressions

Regular expression syntax

Backslashes and raw strings

Character groups and other backslash magic

Nongreedy matching

Extensions

Creating and Using Regular Expression Objects

Using regular expression objects

Applying regular expressions without compiling

Using Match Objects

group([groupid,...])

groups([nomatch])

groupdict([nomatch])

start([groupid]), end([groupid]), span([groupid])

re,string,pos,endpos,

Treating Strings as Files

Encoding Text

Using Unicode strings

Reading and writing non-ASCII strings

Using the Unicode database

Formatting Floating Point Numbers

fix(number,precision)

sci(number,precision)

Chapter 10: Working with Files and Directories

Retrieving File and Directory Information

The piecemeal approach

The I-want-it-all approach

Building and Dissecting Paths

Joining path parts

Breaking paths into pieces

Other path modifiers

Listing Directories and Matching File Names

Obtaining Environment and Argument Information

Environment variables

Current working directory

Command-line parameters

Example: Recursive Grep Utility

Copying, Renaming, and Removing Paths

Copying and linking

Renaming

Removing

Creating Directories and Temporary Files

Comparing Files and Directories

Working with File Descriptors

General file descriptor functions

Pipes

Other File Processing Techniques

Randomly accessing lines in text files

Using memory-mapped files

Iterating over several files

Chapter 11: Using Other Operating System Services

Executing Shell Commands and Other Programs

Spawning Child Processes

popen functions

spawn functions

fork

Process management and termination

Handling Process Information

Retrieving System Information

Managing Configuration Files

Understanding Error Names

Handling Asynchronous Signals

Chapter 12: Storing Data and Objects

Data Storage Overview

Text versus binary

Compression

Byte order ("Endianness")

Object state

Destination

On the receiving end

Loading and Saving Objects

Pickling with pickle

The marshal module

Example: Moving Objects Across a Network

Using Database-Like Storage

Converting to and from C Structures

Converting Data to Standard Formats

Sun's XDR format

Other formats

Compressing Data

zlib

gzip

zipfile

Chapter 13: Accessing Date and Time

Telling Time in Python

Ticks

TimeTuple

Stopwatch time

Converting Between Time Formats

Parsing and Printing Dates and Times

Fancy formatting

Parsing time

Localization

Accessing the Calendar

Printing monthly and yearly calendars

Calendar information

Leap years

Using Time Zones

Allowing Two-Digit Years

Chapter 14: Using Databases

Using Disk-Based Dictionaries

DBM Example: Tracking Telephone Numbers

Advanced Disk-Based Dictionaries

dbm

gdbm

dbhash

Using BSD database objects

Accessing Relational Databases

Connection objects

Transactions

Cursor objects

Example: "Sounds-Like" Queries

Examining Relational Metadata

Example: Creating Auditing Tables

Advanced Features of the DB API

Input and output sizes

Reusable SQL statements

Database library information

Error hierarchy

Part III: Networking and the Internet

Chapter 15: Networking

Networking Background

Working with Addresses and Host Names

Communicating with Low-Level Sockets

Creating and destroying sockets

Connecting sockets

Sending and receiving data

Using socket options

Converting numbers

Example: A Multicast Chat Application

Using SocketServers

The SocketServer family

Request handlers

Processing Web Browser Requests

BaseHTTPRequestHandler

SimpleHTTPRequestHandler

CGIHTTPRequestHandler

Example: form handler CGI script

Handling Multiple Requests Without Threads

asyncore

Chapter 16: Speaking Internet Protocols

Python's Internet Protocol Support

Retrieving Internet Resources

Manipulating URLs

Treating a URL as a file

URLopeners

Extended URL opening

Sending HTTP Requests

Building and using request objects

Sending and Receiving E-Mail

Accessing POP3 accounts

Accessing SMTP accounts

Accessing IMAP accounts

Transferring Files via FTP

Retrieving Resources Using Gopher

Working with Newsgroups

Using the Telnet Protocol

Connecting

Reading and writing

Watching and waiting

Other methods

Writing CGI Scripts

Setting up CGI scripts

Accessing form fields

Advanced CGI functions

A note on debugging

A note on security

Chapter 17: Handling Internet Data

Manipulating URLs

Formatting Text

Formatter interface

Writer interface

Other module resources

Reading Web Spider Robot Files

Viewing Files in a Web Browser

Dissecting E-Mail Messages

Parsing a message

Retrieving header values

Other members

Address lists

rfc822 utility functions

MIME messages

Working with MIME Encoding

Encoding and decoding MIME messages

Parsing multipart MIME messages

Writing out multipart MIME messages

Handling document types

Encoding and Decoding Message Data

Uuencode

Base64

Quoted-printable

Working with UNIX Mailboxes

Working with MH mailboxes

Using Web Cookies

Cookies

Morsels

Example: a cookie importer

Chapter 18: Parsing XML and Other Markup Languages

Markup Language Basics

Tags are for metatext

Tag rules

Namespaces

Processing XML

Parsing HTML Files

HTMLParser methods

Handling tags

Other parsing methods

Handling unknown or bogus elements

Example: Bold Only

Example: Web Robot

 

Parsing XML with SAX

Using a ContentHandler

Example: blood-type extractor

Using parser (XMLReader) objects

SAX exceptions

Parsing XML with DOM

DOM nodes

Elements, attributes, and text

The document node (DOM)

Example: data import and export with DOM

Parsing XML with xmllib

Elements and attributes

XML handlers

Other XMLParser members &

Part IV: User Interfaces and Multimedia

Chapter 19: Tinkering with Tkinter

Getting Your Feet Wet

Creating a GUI

Building an interface with widgets

Widget options

Laying Out Widgets

Packer options

Grid options

Example: Breakfast Buttons

Using Common Options

Color options

Size options

Appearance options

Behavior options

Gathering User Input

Example: Printing Fancy Text

Using Text Widgets

Building Menus

Using Tkinter Dialogs

File dialogs

Example: Text Editor

Handling Colors and Fonts

Colors

Fonts

Drawing Graphics

The canvas widget

Manipulating canvas items

Using Timers

Example: A Bouncing Picture

Chapter 20: Using Advanced Tkinter Widgets

Handling Events

Creating event handlers

Binding mouse events

Binding keyboard events

Event objects

Example: A Drawing Canvas

Advanced Widgets

Listbox

Scale

Scrollbar

Example: Color Scheme Customizer

Creating Dialogs

Supporting Drag-and-Drop Operations

Using Cursors

Designing New Widgets

Further Tkinter Adventures

Additional widgets

Learning more

Chapter 21: Building User Interfaces with wxPython

Introducing wxPython

Creating Simple wxPython Programs

Choosing Different Window Types

Managed windows

Nonmanaged windows

Using wxPython Controls

Common controls

Tree controls

Editor controls

Controlling Layout

Specifying coordinates

Sizers

Layout constraints

Layout algorithms

Using Built-in Dialogs

Drawing with Device Contexts

Adding Menus and Keyboard Shortcuts

Accessing Mouse and Keyboard Input

Other wxPython Features

Clipboard, drag and drop, and cursors

Graphics

Date and time

Fonts

HTML

Printing

Other

Chapter 22: Using Curses

A Curses Overview

Starting Up and Shutting Down

Displaying and Erasing Text

Reading from the window (screen-scraping)

Erasing

Refreshing

Boxes and lines

The window background

Example: masking a box

Moving the Cursor

Getting User Input

Reading keys

Other keyboard-related functions

Fancy characters

Reading mouse input

Example: yes, no, or maybe

Managing Windows

Pads

Stacking windows

Editing Text

Using Color

Numbering

Setting colors

Tweaking the colors

Example: A Simple Maze Game

Chapter 23: Building Simple Command Interpreters

Beginning with the End in Mind

Understanding the Lepto Language

Creating a Lepto Lexical Analyzer

The shlex module

Putting shlex to work

Adding Interactive-Mode Features

Using the cmd module

Subclassing cmd.Cmd

Executing Lepto Commands

Chapter 24: Playing Sound

Sound File Basics

Playing Sounds

Playing sound on Windows

Playing and recording sound on SunOS

Examining Audio Files

Reading and Writing Audio Files

Reading and writing AIFF files with aifc

Reading and writing AU files with sunau

Reading and writing WAV files with wave

Example: Reversing an audio file

Reading IFF chunked data

Handling Raw Audio Data

Examining a fragment

Searching and matching

Translating between storage formats

Manipulating fragments

Part V: Advanced Python Programming

Chapter 25: Processing Images

Image Basics

Identifying Image File Types

Converting Between Color Systems

Color systems

Converting from one system to another

Handling Raw Image Data

Using the Python Imaging Library

Retrieving image information

Copying and converting images

Using PIL with Tkinter

Cropping and resizing images

Modifying pixel data

Other PIL features

Chapter 26: Multithreading

Understanding Threads

Spawning, Tracking, and Killing Threads

Creating threads with the thread module

Starting and stopping threads with the threading module

Thread status and information under threading

Finding threads under threading

Waiting for a thread to finish

Avoiding Concurrency Issues

Locking with thread

Locking with threading

Preventing Deadlock

Example: Downloading from Multiple URLs

Porting Threaded Code

Weaving Threads Together with Queues

Technical Note: How Simultaneous Is Simultaneous?

For More Information

Chapter 27: Debugging, Profiling, and Optimization

Debugging Python Code

Starting and stopping the debugger

Examining the state of things

Setting breakpoints

Running

Aliases

Debugging tips

Working with docstrings

Automating Tests

Synching docstrings with code

Unit testing

Finding Bottlenecks

Profiling code

Using Profile objects

Calibrating the profiler

Customizing statistics

Common Optimization Tricks

Sorting

Looping

I/O

Strings

Threads

Taking out the Trash -- the Garbage Collector

Reference counts and Python code

Reference counts and C/C++ code

Chapter 28: Security and Encryption

Checking Passwords

Running in a Restricted Environment

The rexec sandbox

Using a class fortress

Creating Message Fingerprints 1

MD5

SHA

Other uses

Using 1940s-Era Encryption

Chapter 29: Writing Extension Modules

Extending and Embedding Overview

Writing a Simple Extension Module 8

Building and Linking

Converting Python Data to C

Unpacking normal arguments

Using special format characters

Unpacking keyword arguments

Unpacking zero arguments

Converting C Data to Python

Creating simple Python objects

Creating complex Python objects

Embedding the Interpreter

A simple example

Shutting down

Other setup functions

System information functions

Running Python Code from C

Using Extension Tools

SWIG

CXX

Extension classes

Chapter 30: Embedding the Python Interpreter

Tracking Reference Counts

Types of reference ownership

Reference conventions

Common pitfalls

Using the Abstract and Concrete Object Layers

Object layers

Working with generic objects

Working with Number Objects

Any numerical type

Integers

Longs

Floating-point numbers

Complex numbers

Working with Sequence Objects

Any sequence type

Strings

Lists

Tuples

Buffers

Unicode strings

Working with Mapping Objects

Functions for any mapping type

Dictionaries

Using Other Object Types

Type

None

File

Module

CObjects

Creating Threads and Sub-Interpreters

Threads

Sub-interpreters

Handling Errors and Exceptions

Checking for errors

Signaling error conditions

Creating custom exceptions

Raising warnings

Managing Memory

Chapter 31: Number Crunching

Using Math Routines

Rounding and fractional parts

General math routines

Logarithms and exponentiation

Trigonometric functions

Computing with Complex Numbers

Generating Random Numbers

Random numbers

Example: shuffling a deck

Random distributions

Example: plotting distributions using Monte Carlo sampling

Using Arbitrary-Precision Numbers

Chapter 32: Using NumPy

Introducing Numeric Python

Installing NumPy

Some quick definitions

Meet the array

Accessing and Slicing Arrays

Contiguous arrays

Converting arrays to lists and strings

Calling Universal Functions

Ufunc destinations

Example: editing an audio stream

Repeating ufuncs

Creating Arrays

Array creation functions

Seeding arrays with functions

Using Element Types

Reshaping and Resizing Arrays

Using Other Array Functions

sort(array,[axis=-1])

where(condition,X,Y)

swapaxes(array,axis1,axis2)

Matrix operations 602

Array Example: Analyzing Price Trends

Chapter 33: Parsing and Interpreting Python Code

Examining Tracebacks 605

Printing a traceback -- print_exc and friends

Extracting and formatting exceptions

Example: reporting exceptions in a GUI

Eating arbitrary exceptions is bad for you

Introspection

Review: basic introspection

Browsing classes

Browsing function information

Checking Indentation

Tokenizing Python Code

Example: Syntax-Highlighting Printer

Inspecting Python Parse Trees

Creating an AST

ASTs and sequences

Using ASTs

Low-Level Object Creation

Disassembling Python Code

Part VI: Deploying Python Applications

Chapter 34: Creating Worldwide Applications

Internationalization and Localization

Preparing Applications for Multiple Languages

An NLS example

What it all means

Formatting Locale-Specific Output

Changing the locale

Locale-specific formatting

Properties of locales

Chapter 35: Customizing Import Behavior

Understanding Module Importing

Finding and Loading Modules with imp

Importing Encrypted Modules

Retrieving Modules from a Remote Source

Subclassing Importer

Creating the remote Importer

Testing the remote Importer

Chapter 36: Distributing Modules and Applications

Understanding distutils

Creating a simple distribution

Installing the simple distribution

Other distutils Features

Distributing packages

Including other files

Customizing setup

Distributing Extension Modules

Creating Source and Binary Distributions

Source distributions

Binary distributions

Installers

Building Standalone Executables

py2exe

Freeze

Other tools

Part VII: Platform-Specific Support

Chapter 37: Windows

Using win32all

Data types

Error handling

Finding what you need

Example: Using Some Windows APIs

Accessing the Windows Registry

Accessing the registry with win32all

Example: setting the Internet Explorer home page

Creating, deleting, and navigating keys

Example: recursive deletion of a key

Other registry functions

Accessing the registry with _winreg

Using msvcrt Goodies

Console I/O

Other functions

Chapter 38: UNIX-Compatible Modules

Checking UNIX Passwords and Groups

Accessing the System Logger

Calling Shared Library Functions

Providing Identifier and Keyword Completion

 

Retrieving File System and Resource Information

File system information

Resource usage

Resource limits

Controlling File Descriptors

Handling Terminals and Pseudo-Terminals

Interfacing with Sun's NIS "Yellow Pages"

Appendix A: Online Resources 685

Appendix B: Python Development Environments

Index

Preface