[Python] List Append Performance: += > extend > append

on

I have a Python script that runs a little slowly and I was wondering how I could improve the performance. It is a fairly long for loop where we're trying to find matches in a large list. In the loop we're appending to a list with +=.

Q: Could I improve the performance with .append() or .extend()?

The Test Definition

def test(number):
  from datetime import datetime
  count = 0; a = []
  start = datetime.now)_
  while count != number:
    a += ['TESTING1234']
    #a.append('TESTING1234')
    #a.extend('TESTING1234')
    count += 1
  end = datetime.now()
  print(end - start)
  print(f'Rows in list = {len(a)}')

The Results

I ran the test 3 times for each of +=, .append() and .extend() and the fastest was -

+=

- which I was already using.

The results for test(5000000) were fairly consistent with averages:

  1. += 7 seconds
  2. .extend() 8 seconds
  3. .append() 26 seconds
+= and extend() are very close, but += just edges it. append() is very slow.


THE END

Comments

  1. Stephen6 May 2024 at 05:15

    This is definitely not true, append() appends an element to the end of a list, while you are using += ['TESTING1234'], you are creating a new list and then merging two lists into one for EACH loop, this must be slower than append() based on the amount of work it has done.

    Besides, your usage of extend() is wrong, it should be extend(['TESTING1234']), and += is just a syntactic sugar of extend(), so they are basically the same thing.

    ReplyDelete

Post a Comment