Python for test automation: data types (advanced)

list

List comprehension

There is a Pythonic way to iterate over the elements of a list; list comprehension. It is a concise and efficient way to create lists in Python. It allows you to generate a new list by applying an expression to each element of an existing iterable, such as a list, tuple, or range. The syntax for list comprehension is enclosed in square brackets and consists of an expression followed by a 'for' loop or multiple 'for' loops and optional 'if' conditions. List comprehension is used to streamline code, making it more readable and compact, especially in scenarios where you need to perform simple transformations or filtering operations on a list. It is a powerful tool in Python for creating lists dynamically and is widely used in various programming tasks.

Here is a simple list comprehension creating a copy of a list:

l1 = [0, 1, 2]
l2 = [x for x in l1]

We can add an if-statement to the list comprehension to filter the list:

l1 = [0, 1, 2, 3, 4, 5, 6, 7]
l2 = [x for x in l1 if x % 2 == 0]  # take only even numbers
print(l2)
>>>[0, 2, 4, 6]

We can also modify all (or some) of the elements:

l1 = [0, 1, 2, 3]
l2 = [x**2 for x in l1]  # new list contains the elements
                         # of l1 to the power of 2
print(l2)
>>>[0, 1, 4, 9]

We could do all of this in a traditional for loop, but using a list comprehension is more efficient (faster), and we save some lines of code with the concise syntax of the list comprehension.

Sorting

Sorting a list refers to arranging its elements in a specific order, typically ascending or descending. Python provides a built-in function sorted, and the list method sort to achieve this. Sorting a list is essential for various tasks such as organizing data for easier retrieval, identifying the minimum or maximum values, and facilitating efficient searching algorithms like binary search. It helps in improving the readability and organization of data, making it easier to analyze and manipulate.

sort is a list method, that has a single optional argument key, which is a key function that determines how the items are sorted. By default when calling sort, elements are sorted in an ascending order, e.g. numbers from smallest to greatest, and strings in a alphabetic order.

l = [8, 2, 1, 4, 5, 6]
l.sort()
print(l)
>>>[1, 2, 4, 5, 6, 8]

We can use the key argument to order a list of lists based on their length (instead of the default ordering based on the elements in the lists), by passing key=len to the sort method:

l = [[1, 2, 3], [8], [4, 2, 3, 4], [6, 6, 7], [9], [1]]
l.sort(key=len)
print(l)
>>>[[8], [9], [1], [1, 2, 3], [6, 6, 7], [4, 2, 3, 4]]

We can define custom key-functions as well. We can use a normally defined function, or a lambda function; it is a small, anonymous function defined using the lambda keyword. It is primarily used for short, one-time operations where defining a full function using the def keyword would be cumbersome or unnecessary.

Using a lambda function, we can order the previous list of lists based on which list has the smallest first element:

l = [[1, 2, 3], [8], [4, 2, 3, 4], [6, 6, 7], [9], [1]]
l.sort(key=lambda x: x[0])
print(l)
>>>[[1, 2, 3], [1], [4, 2, 3, 4], [6, 6, 7], [8], [9]]

sort is done in-place, meaning it modifies the list itself. Python has a built-in function sorted which works similarly to sort, but instead of working in-place, it returns an iterator, that can be turned into a list, or iterated using a for loop. sorted has two arguments; the first is the list to sort, and the second named argument is key which has the same usage as with the sort method.

l = [8, 2, 1, 4, 5, 6]
print(list(sorted(l)))
>>>[1, 2, 4, 5, 6, 8]

Since 'sorted' returns an iterator, we first convert it into a list with the built-in 'list' function.

List and tuple splat operator (*)

The splat operator (*), also known as the asterisk operator, is used for unpacking iterables like lists and tuples into separate elements. It allows passing a variable number of arguments to functions or methods, or unpacking sequences into separate elements during assignment. When used in function definitions, the splat operator allows a function to accept any number of positional arguments. Similarly, when used in function calls or method invocations, it allows passing an arbitrary number of arguments to a function.

Here are a few examples on how to use the splat operator.

Adding the elements of a list to another list:

l1 = [0, 1, 2]
l2 = [2, 2, *l1, 3, 3]
print(l2)
>>>[2, 2, 0, 1, 2, 3, 3]

Placing the elements of a list as positional arguments to a function:

def sum_of_three_numbers(a: int, b: int, c: int) -> int:
    return a + b + c


l = [0, 1, 2]
print(sum_of_three_numbers(*l))
>>>3

Double list comprehension

We can also add another for loop to our list comprehension. The syntax of that becomes a bit less readable, but is understandable when the variable names are chosen well:

sentences = ['Here are a few', 'sentences that we can', 'use to demonstrate the', 'double list comprehension in Python']

words = [
    word for sentence in sentences
    for word in sentence.split(" ")
]
print(words)
>>>[
    'Here',
    'are',
    'a',
    'few',
    'sentences',
    'that',
    'we',
    'can',
    'use',
    'to',
    'demonstrate',
    'the',
    'double',
    'list',
    'comprehension',
    'in',
    'Python',
]

Here we use the split method of strings, which splits a string based on the string given as argument, returning a list of strings as a result. Splitting is done on a single space character to separate words.

dict

dict comprehension

Similar to lists, dicts also have a Pythonic way to iterate over the key-value pairs of dict; dict comprehension.

The syntax for dict comprehension is similar to list comprehension, but instead of brackets [] we use curly-braces {}. Here is a simple dict comprehension creating a copy of a dict:

d1 = {"a": 1, "b": 2}
d2 = {k: v for k, v in d1.items()}

Modifying keys and/or values and filtering based on keys or values is a common usecase for dict comprehension.

For example, we can double the value in a key-value pair and filter out odd numbers:

d1 = {"a": 1, "b": 2}
d2 = {k: v * 2 for k, v in d1.items() if v % 2 == 0}
print(d2)
>>>{'b': 4}

This is also an easy way to filter keys based on a list, tuple or a set containing the keys we want:

d1 = {"a": 1, "b": 2, "c": 3}
d2 = {k: v * 2 for k, v in d1.items() if k in {"a", "b"}}
print(d2)
>>>{"a": 1, "b": 2}

Dictionary splat operator (**)

The double asterisk operator (**), also known as the dict splat operator or the keyword argument unpacking operator, is used to unpack dictionaries into separate key-value pairs. It allows passing a variable number of keyword arguments to functions or methods. When used in function definitions, the ** operator collects all keyword arguments into a single dictionary parameter. Conversely, when used in function calls or method invocations, it unpacks a dictionary into individual keyword arguments. This operator is particularly useful when working with functions that require a dynamic set of keyword arguments, as it provides a concise and flexible way to handle such scenarios.

Here are a few examples on how to use the double asterisk operator.

Adding a the keys and values of a dict to another:

d1 = {"a": 1, "b": 2}
d2 = {**d1, "c": 3}
print(d2)
>>>{'a': 1, 'b': 2, 'c': 3}

Placing the key-value pairs of a dict into named arguments of a function:

def sum_of_three_numbers(
    a: int = 0, b: int = 0, c: int = 0
) -> int:
    return a + b + c


d = {"a": 1, "c": 3, "b": 2}
print(sum_of_three_numbers(**d))
>>>6

Initialization using zip

The built-in function zip is used to combine elements from multiple iterables, such as lists, tuples, or strings, into tuples. It iterates over each iterable simultaneously and pairs corresponding elements together. The resulting tuples contain elements from each iterable at the same index position. If the iterables passed to zip have different lengths, the resulting iterator stops as soon as the shortest iterable is exhausted. zip is commonly used in situations where data from multiple sources needs to be processed or manipulated together, such as iterating over multiple lists simultaneously, creating dictionaries from keys and values stored in separate lists, or transposing rows and columns in a matrix-like data structure.

zip is a convenient way to for example read data in a table form to a list of dictionaries, where keys represent column headers and values represent cell values per row.

headers = ["A", "B", "C"]
values = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]

dicts = []
for value in values:
    dicts.append(dict(zip(headers, value)))
print(dicts)
>>>[
    {
        'A': 1,
        'B': 2,
        'C': 3,
    },
    {
        'A': 4,
        'B': 5,
        'C': 6,
    },
    {
        'A': 7,
        'B': 8,
        'C': 9,
    },
]