python querying all rows of azure table

Question

I have around 20000 rows in my azure table . I wanted to query all the rows in the azure table . But due to certain azure limitation i am getting only 1000 rows.

Answer 1

Update 2019

Just running for loop on the query result (as author of the topic does) - will get all the data from the query.

from azure.cosmosdb.table.tableservice import TableService

table_service = TableService(account_name='accont_name', account_key='key')

#counter to keep track of records
counter=0

# get the rows. Debugger shows the object has only 100 records
rows = table_service.query_entities(table,"PartitionKey eq 'mykey'")

for row in rows:
    if (counter%100 == 0):
        # just to keep output smaller, print every 100 records
        print("Processing {} record".format(counter))
    counter+=1 

The output proves that loop goes over a 1000 records

...
Processing 363500 record
Processing 363600 record
...

Answer 2

But due to certain azure limitation i am getting only 1000 rows.

This is a documented limitation. Each query request to Azure Table will return no more than 1000 rows. If there are more than 1000 entities, table service will return a continuation token that must be used to fetch next set of entities (See Remarks section here: http://msdn.microsoft.com/en-us/library/azure/dd179421.aspx)

Please see the sample code to fetch all entities from a table:

from azure import *
from azure.storage import TableService

table_service = TableService(account_name='xxx', account_key='yyy')
i=0
next_pk = None
next_rk = None
while True:
    entities=table_service.query_entities('Address',"PartitionKey eq 'Address'", next_partition_key = next_pk, next_row_key = next_rk, top=1000)
    i+=1
    for entity in entities:
        print(entity.AddressLine1)
    if hasattr(entities, 'x_ms_continuation'):
        x_ms_continuation = getattr(entities, 'x_ms_continuation')
        next_pk = x_ms_continuation['nextpartitionkey']
        next_rk = x_ms_continuation['nextrowkey']
    else:
        break;