system.queues.external_queues module

Disk-backed external queue implementation.

class system.queues.external_queues.ExternalQueue(memory_limit: int = 10000)

Bases: ReadWriteQueue, Generic

ExternalQueue is a disk-backed read-write queue that buffers elements in memory and spills to a temporary file once the buffer is full.

reset() → None

Resets the queue to the initial state.

pop() → ObjectType

Not supported by this disk-backed queue: always raises NotImplementedError.

insert(entry: ObjectType) → None

Inserts an entry into the queue

Parameters:

entry – The entry to insert into the queue.

flush() → int

Flushes the buffer to the file, i.e. writes the contents of the buffer to the file and empty the buffer.

close() → None

Closes the temporary file backing the queue. This will also delete that file.

size() → int

Returns the number of elements inserted into this queue so far