Python: How to append a new row to an existing csv file?

In this article, we will discuss how to append a row to an existing csv file using csv module’s reader / writer & DictReader / DictWriter classes.

Suppose we have a CSV file students.csv, whose contents are,

Id,Name,Course,City,Session
21,Mark,Python,London,Morning
22,John,Python,Tokyo,Evening
23,Sam,Python,Paris,Morning

There can be many ways in python, to append a new row at the end of this csv file. But here we will discuss two effective ways using csv module.

Python provides a csv module for reading and writing csv files. For writing csv files, it has two different classes i.e. writer and DictWritter.

We can append a new line in csv by using either of them. But they are some scenarios which makes one solution better than other. So, let’s discuss both of them one by one,

Append a list as new row to an old csv file using csv.writer()

csv module provides a class writer, which can write lists into csv file as rows. Let’s see how to use it for appending a new row in csv,

Suppose we have a list of strings,

# List of strings
row_contents = [32,'Shaun','Java','Tokyo','Morning']

To add the contents of this list as a new row in the csv file, we need to follow these steps,

• Import csv module’s writer class,
• Open our csv file in append mode and create a file object.
• Pass this file object to the csv.writer(), we can get a writer class object.
• This writer object has a function writerow() , pass the list to it and it will add list’s contents as a new row in the associated csv file
• As new row is added in the csv file, now close the file object.

The above steps will append out list as a row in the csv. To make the process simple, we have created a separate function with the above steps,

from csv import writer

def append_list_as_row(file_name, list_of_elem):
    # Open file in append mode
    with open(file_name, 'a+', newline='') as write_obj:
        # Create a writer object from csv module
        csv_writer = writer(write_obj)
        # Add contents of list as last row in the csv file
        csv_writer.writerow(list_of_elem)

This function accepts a csv file name and a list of elements as arguments. Then adds the content of list as a new row in the end of csv file. Let’s use this function to append a row in our csv file ‘students.csv’,

# List of strings
row_contents = [32,'Shaun','Java','Tokyo','Morning']

# Append a list as new line to an old csv file
append_list_as_row('students.csv', row_contents)

Now the contents of the file ‘students.csv’ are,

Id,Name,Course,City,Session
21,Mark,Python,London,Morning
22,John,Python,Tokyo,Evening
23,Sam,Python,Paris,Morning
32,Shaun,Java,Tokyo,Morning

It added the list as row in the existing csv file ‘students.csv’.

Appending a row to csv with missing entries ?

What If the size of out list is less than the number of columns in csv file i.e. if some entries are missing in the list ?

Check out this example,

# A list with missing entries
row_contents = [33, 'Sahil', 'Morning']

# Appending a row to csv with missing entries
append_list_as_row('students.csv', row_contents)

Now the contents of the student.csv are,

Id,Name,Course,City,Session
21,Mark,Python,London,Morning
22,John,Python,Tokyo,Evening
23,Sam,Python,Paris,Morning
32,Shaun,Java,Tokyo,Morning
33,Sahil,Morning

As in list their were fewer items , so it appended a new row, but values were added in the wrong columns like ‘Morning’ was added in column ‘Course’.

csv’s writer class has no functionality to check if any of the intermediate column values are missing in the list or if they are in the correct order. It will just add the items in the list as column values of the last row in sequential order.

Therefore while adding a list as a row using csv.writer() we need to make sure that all elements are provided and in correct order.

If any element is missing like in the above example, then we should pass empty strings in the list like this,

row_contents = [33, 'Sahil', '' , '', 'Morning']

But if we have a huge number of columns in our csv file and most of them are empty then creating empty entries for each of them will be a hectic task. To save us from this hectic work, csv module provides an another class DictWriter. Let’s see how we can use it.

Append a dictionary as a row to an existing csv file using DictWriter in python

csv module provides a class DictWriter, which can write dictionaries into csv file as rows. Let’s see how to use it for appending a new row in csv,

Suppose we have a dictionary,

{'Id': 81,'Name': 'Sachin','Course':'Maths','City':'Mumbai','Session':'Evening'}

Keys in this dictionary matches with the column names in csv file. To add the contents of this dictionary as a new row in the csv file, we need to follow these steps,

• import csv module’s DictWriter class,
• Open our csv file in append mode and create a file object,
• Pass the file object & a list of csv column names to the csv.DictWriter(), we can get a DictWriter class object
• This DictWriter object has a function writerow() that accepts a dictionary. pass our dictionary to this function, it adds them as a new row in the associated csv file,
• As new line is added in the csv file, now close the file object,

The above steps will append our dictionary as a new row in the csv. To make our life easier, we have created a separate function that performs the above steps,

from csv import DictWriter

def append_dict_as_row(file_name, dict_of_elem, field_names):
    # Open file in append mode
    with open(file_name, 'a+', newline='') as write_obj:
        # Create a writer object from csv module
        dict_writer = DictWriter(write_obj, fieldnames=field_names)
        # Add dictionary as wor in the csv
        dict_writer.writerow(dict_of_elem)

This function accepts 3 arguments i.e. a csv file name, dictionary of elements and fields names. Then appends the content of dictionary as a new row in the end of csv file.

Let’s look at an example,

Contents of the csv file ‘students.csv’ are,

Id,Name,Course,City,Session
21,Mark,Python,London,Morning
22,John,Python,Tokyo,Evening
23,Sam,Python,Paris,Morning
32,Shaun,Java,Tokyo,Morning

Use the above created function to append a dictionary as a row in our csv file ‘students.csv’,

field_names = ['Id','Name','Course','City','Session']
row_dict = {'Id': 81,'Name': 'Sachin','Course':'Maths','City':'Mumbai','Session':'Evening'}

# Append a dict as a row in csv file
append_dict_as_row('students.csv', row_dict, field_names)

Now the contents of the file ‘students.csv’ are,

Id,Name,Course,City,Session
21,Mark,Python,London,Morning
22,John,Python,Tokyo,Evening
23,Sam,Python,Paris,Morning
32,Shaun,Java,Tokyo,Morning
81,Sachin,Maths,Mumbai,Evening

It added the row successfully. But what if our row has missing entries ?

Appending a dictionary as a row to csv with missing entries ?

If there are missing entries in our dictionary or even items are in different order, in both the cases DictWriter class will automatically handle it and will add only given columns in the new row. Column with missing entries will be empty.

Check out this example,

# If empty entries are missed then DictWriter will handle them automatically
field_names = ['Id','Name','Course','City','Session']
row_dict = {'Id': 33, 'Name':'Eva', 'Session':'Evening'}

# Append a dict missing entries as a row in csv file
append_dict_as_row('students.csv', row_dict, field_names)

Now the contents of the student.csv are,

Id,Name,Course,City,Session
21,Mark,Python,London,Morning
22,John,Python,Tokyo,Evening
23,Sam,Python,Paris,Morning
32,Shaun,Java,Tokyo,Morning
81,Sachin,Maths,Mumbai,Evening
33,Eva,,,Evening

As in dictionary their were values for keys ‘Id’, ‘Name’ & ‘Session’ only. So, DictWriter added a row with these columns only and that too in correct position.

The complete example is as follows,

from csv import writer
from csv import DictWriter

def append_list_as_row(file_name, list_of_elem):
    # Open file in append mode
    with open(file_name, 'a+', newline='') as write_obj:
        # Create a writer object from csv module
        csv_writer = writer(write_obj)
        # Add contents of list as last row in the csv file
        csv_writer.writerow(list_of_elem)


def append_dict_as_row(file_name, dict_of_elem, field_names):
    # Open file in append mode
    with open(file_name, 'a+', newline='') as write_obj:
        # Create a writer object from csv module
        dict_writer = DictWriter(write_obj, fieldnames=field_names)
        # Add dictionary as wor in the csv
        dict_writer.writerow(dict_of_elem)


def main():
    print('*** Append new row to an existing csv file using csv.writer() in python ***')

    # List of strings
    row_contents = [32,'Shaun','Java','Tokyo','Morning']

    # Append a list as new line to an old csv file
    append_list_as_row('students.csv', row_contents)

    print('** Append new row to an existing csv file with less items using csv.writer()  in python **')

    # A list with missing entries
    row_contents = [33, 'Sahil', 'Morning']

    # Appending a row to csv with missing entries
    append_list_as_row('students.csv', row_contents)

    print('Append dictionary as a row to an existing csv file using DictWriter in python')

    field_names = ['Id','Name','Course','City','Session']
    row_dict = {'Id': 81,'Name': 'Sachin','Course':'Maths','City':'Mumbai','Session':'Evening'}

    # Append a dict as a row in csv file
    append_dict_as_row('students.csv', row_dict, field_names)

    print('Append a dictionary with missing entries as a new row to an existing csv file using DictWriter**')

    # If empty entries are missed then DictWriter will handle them automatically
    field_names = ['Id','Name','Course','City','Session']
    row_dict = {'Id': 33, 'Name':'Eva', 'Session':'Evening'}

    # Append a dict missing entries as a row in csv file
    append_dict_as_row('students.csv', row_dict, field_names)


if __name__ == '__main__':
    main()