Variables, Objects, and References in Python

Basics

When you run the following code: 
>>> x = 2
The following will take place in Python:
  1. (Object creation) An object with value 2 will be created.
  2. (Variable creation) If a variable with name x doesn't exist, variable x will be created.
  3. (Reference) The variable x will refer to object 2.
More interestingly, the followign code works fine: 
>>> x = 2
>>> type(x)
<class 'int'>
>>> x = "hello"
>>> type(x)
<class 'str'>
To make sense out of this, remember:

Shared references

What will happen if the following code is executed? 
>>> x = 2
>>> y = x  ## focus here
Variable y will refer to what x refers to, as shown below:
  

Referring to something else

Of course, we can have variable x refer to something else as follows: 
>>> x = 2
>>> y = x  
>>> x = "hello"   ## focus here
The results is shown below:

Lists and Tuples

Lists are sequences of items of any type, when specified in an assignment statement the list starts with symbol '[' and items are separated by commas ending in ']', for example: 

>>> A = ["hello", "world", "how", "are", "you", "?"]  # a list of strings.
>>> B = [42, 1, 2, 3, 4, 5]                           # a list of integers.
>>> C = [1,"hi","world",4, None]                      # a list of objects (mixed types) 
>>> D = []                                            # an empty list
>>> E = [None, [], [1], [1,2], [1,2,3] ]              # a list of None type and lists. 
>>> F = [[1]]                                         # a list whose first element is a list.
>>> type(A)
<class 'list'>                                        # type is class 'list'
>>> len(A)                                            # returns the number elements in the list 
6

Indexing in a list

You can access each individual item using the indexing operator. 

>>> C
[1, 'hi', 'world', 4, None]
>>> C[2] = [1,2]                                   
>>> C
[1, 'hi', [1, 2], 4, None]
>>> C[-1]
>>> C[-2]
4
>>> C[-3]
[1, 2]
>>> L = C[2:4]
>>> L
[[1, 2], 4]

Other list operations

There are many other useful operations. In fact, Python provides almost all list/array operations you can think of. Check the documentation as necessary.

Tuples

A tuple is very similar to a list in that it is a sequence of values, except that it is immutable and is constructed using parentheses instead of square brackets. 

>>> myTuple = ('blue','yellow')    # create a tuple 
>>> type(myTuple)
<class 'tuple'>
>>> print(myTuple)
('blue', 'yellow')
>>> print(myTuple[1])              # accessing values in a tuple yellow
yello
>>> myTuple[1] = 'white'           # not allowed to change the item: immutable!
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  TypeError: 'tuple' object does not support item assignment

end Option in print()

You can have print() output whatever you want in the end. By default, print() outputs a new line. 
# ex2.py
print("hello")
print("world")
print("IT432")

print("hello", end="")
print("world", end="!!")
print("IT432")

$ python3 ex2.py
hello
world
IT432
helloworld!!IT432

Loops and Lists/Tuples

in operator

Python lists take advantage of the in operator in a couple of ways. First, this operation can be used to check if an object belongs to a given list. 

>>> myList = [2.4,7.6,243.0,56.5,10.8]
>>> 10 in myList
False
>>> 10.8 in myList
True
You can also use the in operator with a for loop: 

>>> for x in myList: print(x)
...
2.4
7.6
243.0
56.5
10.8

range(start, end)

The range() function is useful to visit all necessary indices. 

>>> for i in range(0,len(myList)): print(i)
... 
0
1
2
3
4
>>> for i in range(0, len(myList)): print(myList[i])
...
2.4
7.6
243.0
56.5
10.8

Comprehension

List comprehensions are a way to build a new list by applying an expression to each item in a sequence, and are close relatives to for loops. 

>>> L = [i for i in range(10)]
>>> L
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> ['a'*i for i in range(5)]
['', 'a', 'aa', 'aaa', 'aaaa']
>>> [chr(ord('a')+i) for i in range(5)]    # ord() returns ASCII code
['a', 'b', 'c', 'd', 'e']                  # chr() returns a string for the ASCII code

Type conversion: string ↔ int

input(), strings, and integers

The built-in function input() takes the keyboard input from the user (stopping when it sees '\n'). 

>>> s = input("Enter: ")
Enter:  This is a test.


>>> s
'This is a test.'
>>> len(s)
15
>>> s[0]
'T'
>>> type(s)
<class 'str'>
>>> type(s[0])
<class 'str'>
>>> s += 'go go go'
>>> s
'This is a test.go go go'

split() and other operations

One common operation on strings is splitting a given string, usually based on whitespace. There are many other useful operations. Check the documentation. 

>>> L = s.split()
>>> L
['This', 'is', 'a', 'test.go', 'go', 'go']
>>> s.lower()
'this is a test.go go go'
>>> 'test' in s
True
>>> 'dest' in s
False
>>> s*3
'This is a test.go go goThis is a test.go go goThis is a test.go go go'
>>> s+s
'This is a test.go go goThis is a test.go go go'

String to integer

Note that input() returns a string. So, if you expect a number from the user, you have to convert the string to an integer. 

>>> n = input("Enter a number: ")
Enter a number: 1000
>>> n %= 35
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: not all arguments converted during string formatting
>>> n = int(n)
>>> n %= 35
>>> n
20

Other operations


>>> n = 20
>>> n // 5			# ingeger division (with truncation)
4
>>> n / 5                       # floating point division
4.0
>>> type(4.0)
<class 'float'>
>>> s = str(4.0)                # type conversion
>>> s
'4.0'
>>> type(s)
<class 'str'>
Note how the operators // and / work differently.

Functions

Use keyword def to define a function. 
>>> def pr0(a): 
...   print(a[0])
...

Recall: Variables are just names.

We stress that variables are just names. Check out the following. 

>>> pr0("hello")
h
>>> pr0([1,2,3,4])
1
>>> pr0(b"1202")
49
In the above, the function pr0 works for any type of objects. In the end, variable a is just a name referring to an object. Make sure to fully understand what's happening in the above.

Dictionary

A dictionary is a data structure that represents a key-value store. 

>>> myDict = {}             # a dictionary uses curly braces instead of square brackets
>>> myDict['hello'] = 'a'   # adds a key:value pair of 'hello':'a'
>>> myDict[1] = 'b'         # adds a key:value pair of 1:'b'
>>> myDict[3.3] = 'c'       # adds a key:value pair of 3.3:'c'
>>> print(myDict['hello'])  # queries what the value associated/mapped to the key 'hello'
a
>>> print(myDict)           # shows us all of the key:value pairs inside of our dictionary 
{'hello': 'a', 1: 'b', 3.3: 'c'}

Looping through a Dictionary

You can use the following member functions to access all the keys and all the values of a given dictionary. 

>>> myDict = {'hello': 'a', 1: 'b', 3.3: 'c'}
>>> ks = myDict.keys()
>>> ks
dict_keys(['hello', 1, 3.3])
>>> myDict['xxx'] = 'yyy'
>>> ks                                      # note: ks immediately sees the change in myDict
dict_keys(['hello', 1, 3.3, 'xxx'])
>>> vs = myDict.values()
>>> vs
dict_values(['a', 'b', 'c', 'yyy'])
>>> for k in ks: print(k)
...
hello
1
3.3
xxx
>>> for v in vs: print(v)
...
a
b
c
yyy

Comprehension

You can create a dictionary using comprehension. 

>>> D = { x: x**2 for x in range(5) }
>>> D
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
>>> L = {0, -1, 'hello', 'x'}
>>> D = { x: x+1 if type(x) == int else x+'!' for x in L }
>>> D
{0: 1, 'hello': 'hello!', 'x': 'x!', -1: 0}
In the above,

Other operations

See the documentation for more detail. Common operations are:

bytes type

In this course, you will often need to look at low-level bits and bytes. In this lecture, we mainly review how to inspect and manipulate bits and bytes of the data.

The bytes object works exactly like an array of bytes. That is, each member a[i] in a bytes object a is a number (0-255). This type is used when we have to deal with raw bytes in python. 

>>> a = b"hello world"
>>> type(a)                 # note the difference in types of a and b
<class 'bytes'>
>>> a[0]
104
>>> type(a[0])              # note the difference in types of a[0] and b[0] as well!!
<class 'int'>
>>> b = "hello world"
>>> type(b)
<class 'str'>
>>> b[0]
'h'
>>> type(b[0])
<class 'str'>

bytearray

Type bytes creates an immutable object. That is, it is read-only, and you cannot change it. If you want to change the contents, you need to use a bytearray object. 
>>> a = b"hello world"
>>> a[0]
104
>>> a[1] = 77                       # immutable!
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'bytes' object does not support item assignment
>>> a = bytearray(b"hello world")
>>> a[0]
104
>>> a[0] = 77                       # you can change it!
>>> a
bytearray(b'Mello world')           # see!
>>> a[0] = 300                      # each byte should be between 0 and 255!
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: byte must be in range(0, 256)

Parameter Passing in Python

The reference model in python leads to some pecularities in passing parameters to a function. We will try to understand this by inspecting the following code: 
1: def reassign(list):
2:   list = [0, 1]
3:
4: def append(list):
5:   list.append(1)
6:
7: list = [0]
8: reassign(list)
9: append(list)

Scope

First of all, each function has a separate scope. That is:
  • Variable list in line 7 belongs to the global scope.
  • However, variable list in lines 1-2 belongs to scope of the function reassign().
  • Likewise, variable list in lines 4-5 belongs to scope of the function append().
In summary, the above code can be re-written as on the right. Since it's less confusing, from now on, we will use that code. 
1: def reassign(l):   ### name change
2:   l = [0, 1]
3:
4: def append(l):   ### name change
5:   l.append(1)
6:
7: list = [0]
8: reassign(list)
9: append(list)

Line 7

Now, let's consider line 7, which will result in the following reference structure:

Line 1

The function call in line 8 will move the control to line 1. At this moment, Python performs l = list so that the function uses the local variable l.

Line 2

Now, in line 2, the variable refers to newly created object [0, 1].

Unfortunately, variable list in the global scope is never changed. 		  

Line 9 then 4

Now, the function reassign returns to line 9. The function call in line 9 will move the control to line 4. Python performs l = list so that the function uses the local variable l.

Line 5

Now, l.append(1) will append 1 at the end of the object that l refers to.

Note. Since list refers to the same object, list refers to object [0,1], now!
Please read the above explanations carefully so that you may fully understand how the two functions work differently.

Using import in python3

When using the code in other python file, you can use the "import" keyword.

Consider a file hello.py. 

# hello.py 
def func():
    print("hello world IT430")

def func2():
    print("Go Navy!")
There are three ways to call the function hello().

Method 1: Simple import

>>> import hello
>>> hello.func()
hello world IT430
>>> hello.func2()
Go Navy!
As shown above:

Method 2: from module import object

If you have to write "hello." every time, it may be cumbersome. So, if you know you access the object many times, you can do as follows: 
>>> from hello import func    ## note here
>>> func()
hello world IT430
>>> func2()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  NameError: name 'func2' is not defined
>>> from hello import func2
>>> func2()
Go Navy!
In the above, first call to func2() failed, since the first from/import allowed you to acces only func().

Method 3: from module import *

By writing "import *", you can import all objects at one shot. 
>>> from hello import *    ## note here
>>> func()
hello world IT430
>>> func2()
Go Navy!

Reading File

Reading a file in strings

You can read files using open() and readlines(). 

# ex.py
import sys

try:
    f = open(sys.argv[1], "r")
    lines = f.readlines()
    print("lines:")
    print("  ", lines)
    print("the first line splitted: ")
    print("  ", lines[0].split())
except OSError:
    print("File not found!")
    exit(0)
data.txt 
hello world
this is 1
it430 100 test
Example runs: 
$ python3 ex.py xxx
File not found!

$ python3 ex.py data.txt
lines:
  ['hello world\n', 'this is 1\n', 'it430 100 test\n']
the first line splitted:
  ['hello', 'world']

Reading a file in raw bytes

You can read the raw bytes of a file data by opening the file in the binary mode. 
>>> f = open("ex2.py", "rb")  # "rb" means "read" in the "binary" mode
>>> b = f.read()
>>> b
b'# ex2.py\nprint("hello")\nprint("world")\nprint("IT430")\n\nprint("hello", end="")\nprint("world", end="!!")\nprint("IT430")\n'
>>> type(b)
<class 'bytes'>
>>> len(b)
118

Bitwise Operators

Formatted string in Python

Here, we generally overview the topic so we can use it for our simple purpose. More details can be found in Section 7.1.2 in this page.
  • It's quite similar to printf in the C programming language.
  • That is, the print code in pr() function on the right reads like the following in C: 
    printf("dec: %d\nhex: 0x%x\nbin: %b", a, a, a);
    Unfortunately, the C programming language doesn't have %b format-specifier. So, the above doesn't really compile.
  • Main point: roughly, think of {:d} as %d in C. So, {a:d} prints the variable a as an integer.
  • The expression 
    f"dec: {a:d}\nhex: 0x{a:02x}\nbin: {a:04b}"
    actually evaluates a string.
  • {:02x} means
    • 0: pad 0 in front if necessary.
    • 2: make sure the output has at least 2 letters.
    • x: in hexadecimal format
    So, because of "02", the number 3 is printed as "03"; that is, two letters and 0 is padded in front.
  • Likewise, 04b means print a number in binary (b) using at least 4 letters while padding 0 in front if necessary. 
>>> def pr(a):
...   print(f"dec: {a:d}\nhex: 0x{a:02x}\nbin: {a:04b}")
...
>>> pr(0x3)
dec: 3
hex: 0x03
bin: 0011
>>> pr(3)
dec: 3
hex: 0x03
bin: 0011
>>> pr(0b0011)
dec: 3
hex: 0x03
bin: 0011
>>> pr(0b1101 & 0b1010)
dec: 8
hex: 0x08
bin: 1000
>>> pr(0b1101 | 0b1010)
dec: 15
hex: 0x0f
bin: 1111
>>> pr(0b1101 ^ 0b1010)
dec: 7
hex: 0x07
bin: 0111
>>> pr(0b1101 >> 1)
dec: 6
hex: 0x06
bin: 0110
>>> pr(0b1101 << 1)
dec: 26
hex: 0x1a
bin: 11010

Manipulating bits (Important)

Question

You are given an integer x that is an 8-bit number (i.e., x is between 0 and 255). How would you reset the 2nd last bit?

Answer (Drag your mouse): 

x &= 0xfd

Practice Problem

Write a program ex.py that works like this: 

~/$ python3 ex.py
X = [1.1, 2.2, 3.3]
Y = [1, -1, 0.1]
<X,Y> = 1.1*1.0 + 2.2*(-1.0) + 3.3*0.1 = -0.77

~/$ python3 ex.py
A = [-1.2]
B = [-4.3]
<A,B> = (-1.2)*(-4.3) = 5.159999999999999

Solutions

See ex.py.