Python for test automation: files

File handling is facilitated through built-in functions and methods provided by the open function. To read or write to a file, one typically starts by opening it using open with the desired file mode ('r' for reading, 'w' for writing, 'a' for appending, etc.). Once opened, the file object provides various methods such as read, write, readline, and close for interacting with the file's contents. Reading from a file involves methods like read to retrieve the entire content, or readline to read one line at a time. Similarly, writing to a file can be achieved with the write method. It's essential to close the file using the close method after all operations are complete to release system resources. Python also supports file handling in binary mode ('rb', 'wb', etc.) for dealing with non-text files like images or executables. Additionally, the os and shutil modules offer functionalities for file manipulation and management, such as renaming, deleting, or moving files.

Here to open and close a file:

f = open("file.txt", "r")
for line in f:
    print(line)
f.close()

Instead of manually closing the file, it is good practice to always use the with-statement to handle opening and closing the file:

with open("file.txt", "r") as f:
    for line in f:
        print(line)

Here we do not need to manually handle closing the file; that is handled automatically after the indented block has completed running. Using the with-statement also ensures that the file is closed even when an exception is raised by the code running inside the statement.

CSV files

The comma-separated values file format can be used to store test data in a test automation project. Handling CSV files is streamlined through the built-in csv module, offering functionalities for reading from and writing to CSV files. To begin, one typically imports the csv module and utilizes its reader and writer objects to handle CSV file data. For reading, the reader object facilitates easy iteration through each row of the CSV file, with options to specify delimiters, quote characters, and other formatting options. Conversely, the writer object enables efficient writing of data to a CSV file, with similar customization options. Additionally, the DictReader and DictWriter classes allow for working with CSV files where each row is represented as a dictionary, offering more intuitive data manipulation. Python's CSV module seamlessly handles edge cases like handling escaped characters and empty fields, ensuring data integrity.

reader

To read a csv file, we first open it with the open function, then pass the opened file to the reader function in the csv package, which in turn returns a reader object that we can iterate over to go through each line in the csv as a list of values.

import csv


with open("file.csv", "r") as f:
    reader = csv.reader(f)
    for line in reader:
        print(line)
>>>['a', 'b', 'c']
>>>['1', '2', '3']
>>>['4', '5', '6']
>>>['7', '8', '9']

writer

To write a csv file, we first open it with the open function, then pass the opened file to the reader function in the csv package, which in turn returns a writer object that we can use to write individual rows or multiple rows at a time:

import csv


with open("out.csv", "w") as f:
    writer = csv.writer(f)

    # write a single row
    writer.writerow(
        ["a", "b", "c"],
    )

    # write multiple rows at once
    writer.writerows(
        [
            [1, 2, 3],
            [4, 5, 6],
            [7, 8, 9],
        ]
    )

DictReader

To read a csv file into a list of dictionaries where each dict in the list has key-value pairs so that the keys match the headers in the csv, and the values match a single row in the csv:

import csv


with open("test.csv", "r") as f:
    reader = csv.DictReader(f)
    for row in reader:
        print(row)
>>>{'a': '1', 'b': '2', 'c': '3'}
>>>{'a': '4', 'b': '5', 'c': '6'}
>>>{'a': '7', 'b': '8', 'c': '9'}

DictWriter

To write a csv file from a list of dictionaries, we can use DictWriter:

import csv


with open("out.csv", "w") as f:
    data = [
        {
            "a": 1,
            "b": 2,
            "c": 3,
        },
        {
            "a": 4,
            "b": 5,
            "c": 6,
        },
        {
            "a": 7,
            "b": 8,
            "c": 9,
        },
    ]

    # here we specify the fieldnames (header columns) to be the
    # keys ('a', 'b' and 'c') found in the first dict in our
    # data list
    writer = csv.DictWriter(f, fieldnames=data[0].keys())

    # write the headers into the opened file
    writer.writeheader()

    # write the data rows
    writer.writerows(data)

JSON files

Handling JSON files is straightforward thanks to the built-in json module, which provides functionalities for encoding and decoding JSON data. Typically, JSON files are read using the json.load function, which parses the JSON data from a file object into Python data structures like dictionaries or lists. Similarly, to write Python data back into a JSON file, the json.dump function is employed, converting Python objects into their JSON representation and storing them in the specified file. Alternatively, json.loads and json.dumps functions can be used to work with JSON strings instead of file objects.

load

To read a JSON file into a Python dictionary, we first use the open function to open the JSON file for reading, then pass the opened file object to the json.load() function which returns a dictionary or list of dictionaries depending on the content of the JSON file. If we have a JSON file that contains

{
  "a": 1,
  "b": "hello",
  "c": [1, 2, 3],
  "d": {
    "e": "hi"
  }
}

then we can read that file into a Python dictionary as follows:

import json


with open("file.json", "r") as f:
    data = json.load(f)
    print(data)
>>>{'a': 1, 'b': 'hello', 'c': [1, 2, 3], 'd': {'e': 'hi'}}

dump

Conversely, we can dump the contents of a dictionary into a JSON file (i.e. if the contents of the dictionary are fully JSON encodeable; only strings, numbers, booleans, None (null), and lists and dictionaries containing those values are accepted). Given the dictionary from our previous example {'a': 1, 'b': 'hello', 'c': [1, 2, 3], 'd': {'e': 'hi'}}, we can dump it into a JSON file using the json.dump() function as follows:

import json

d = {'a': 1, 'b': 'hello', 'c': [1, 2, 3], 'd': {'e': 'hi'}}
with open("out.json", "w") as f:
    json.dump(d, f)

Formatting

When dumping a dictionary into a JSON file, by default it will be written on a single line, which is not ideal for a human to read the contents. We can use the indent argument of json.dump to format the JSON nicely. For the previous example, if we add indent=2 to the json.dump call, we get the following output:

{
  "a": 1,
  "b": "hello",
  "c": [
    1,
    2,
    3
  ],
  "d": {
    "e": "hi"
  }
}