Kamaelia docs : Kamaelia.File.Reading

Components for reading from files

These components provide various ways to read from files, such as manual control of the rate at which the file is read, or reusing a file reader to read from multiple files.

Key to this is a file reader component that reads data only when asked to. Control over when data flows is therefore up to another component - be that a simple component that requests data at a constant rate, or something else that only requests data when required.

PromptedFileReader

This component reads bytes or lines from the specified file when prompted.

Send the number of bytes/lines to the "inbox" inbox, and that data will be read and sent to the "outbox" outbox.

Example Usage

Reading 1000 bytes per second in 10 byte chunks from 'myfile':

Pipeline(ByteRate_RequestControl(rate=1000,chunksize=10)
         PromptedFileReader("myfile", readmode="bytes")
        ).activate()

More detail

The component will terminate if it receives a shutdownMicroprocess message on its "control" inbox. It will pass the message on out of its "signal" outbox.

If unable to read all N bytes/lines requested (perhaps because we are nearly at the end of the file) then those bytes/lines that were read successfully are still output.

When the end of the file is reached, a producerFinished message is sent to the "signal" outbox.

The file is opened only when the component is activated (enters its main loop).

The file is closed when the component shuts down.

RateControlledFileReader

This component reads bytes/lines from a file at a specified rate. It is performs the same task as the ReadFileAdapter component.

You can configure the rate, and the chunk size or frequency.

Example Usage

Read 10 lines per second, in 2 chunks of 5 lines, and output them to the console:

Pipeline(RateControlledFileReader("myfile", "lines", rate=10, chunksize=5),
         ConsoleEchoer()
        ).activate()

More detail

This component is a composition of a PromptedFileReader component and a ByteRate_RequestControl component.

The component will shut down after all data is read from the file, emitting a producerFinished message from its "signal" outbox.

The component will terminate if it receives a shutdownMicroprocess message on its "control" inbox. It will pass the message on out of its "signal" outbox.

The inbox "inbox" is not wired and therefore does nothing.

ReusableFileReader

A reusable PromptedFileReader component, based on a Carousel component. Send it a new filename and it will start reading from that file. Do this as many times as you like.

Send it the number of bytes/lines to read and it will output that much data, read from the file.

Example Usage

Read data from a sequence of files, at 1024 bytes/second in 16 byte chunks:

playlist = Chooser(["file1","file2","file3", ...]
rate = ByteRate_RequestControl(rate=1024,chunksize=16)
reader = ReusableFileReader("bytes")

playlist.link( (reader, "requestNext"), (playlist,"inbox") )
playlist.link( (playlist,"outbox"), (reader, "next") )

Pipeline(ratecontrol, reader).activate()

Or, with the Control-Signal path linked up properly, using the JoinChooserToCarousel prefab:

playlist = Chooser(["file1","file2","file3", ...]
rate = ByteRate_RequestControl(rate=1024,chunksize=16)
reader = ReusableFileReader("bytes")

playlistreader = JoinChooserToCarousel(playlist, reader)

Pipeline(ratecontrol, playlistreader).activate()

More detail

Bytes or lines are read from the file on request. Send the number of bytes/lines to the "inbox" inbox, and that data will be read and sent to the "outbox" outbox.

This component will terminate if it receives a shutdownMicroprocess or producerFinished message on its "control" inbox. The message will be passed on out of its "signal" outbox.

No producerFinished or shutdownMicroprocess type messages are sent by this component between one file and the next.

RateControlledReusableFileReader

A reusable file reader component, based on a Carousel component. Send it a filename and the rate you want it to run at, and it will start reading from that file at that rate. Do this as many times as you like.

Example Usage

Read data from a sequence of files, at different rates:

playlist = Chooser([ ("file1",{"rate":1024}),
                     ("file2",{"rate":16}), ...])
reader = RateControlledReusableFileReader("bytes")

playlist.link( (reader, "requestNext"), (playlist,"inbox") )
playlist.link( (playlist,"outbox"), (reader, "next") )

reader.activate()
playlist.activate()

Or, with the Control-Signal path linked up properly, using the JoinChooserToCarousel prefab:

playlist = Chooser([ ("file1",{"rate":1024}),
                     ("file2",{"rate":16}), ...])
reader = RateControlledReusableFileReader("bytes")

playlistreader = JoinChooserToCarousel(playlist, reader).activate()

More detail

The rate control is performed by a ByteRate_RequestControl component. The rate arguments should be those that are accepted by this component.

This component will terminate if it receives a shutdownMicroprocess or producerFinished message on its "control" inbox. The message will be passed on out of its "signal" outbox.

No producerFinished or shutdownMicroprocess type messages are sent by this component between one file and the next.

FixedRateControlledReusableFileReader

A reusable file reader component that reads data from files at a fixed rate. It is based on a Carousel component.

Send it a new filename and it will start reading from that file. Do this as many times as you like.

Example Usage

Read data from a sequence of files, at 10 lines a second:

playlist = Chooser(["file1", "file2", "file3", ... ])
reader = FixedRateControlledReusableFileReader("lines", rate=10, chunksize=1)

playlist.link( (reader, "requestNext"), (playlist,"inbox") )
playlist.link( (playlist,"outbox"), (reader, "next") )

reader.activate()
playlist.activate()

Or, with the Control-Signal path linked up properly, using the JoinChooserToCarousel prefab:

playlist = Chooser(["file1", "file2", "file3", ... ])
reader = FixedRateControlledReusableFileReader("lines", rate=10, chunksize=1)

playlistreader = JoinChooserToCarousel(playlist, reader).activate()

More detail

The rate control is performed by a ByteRate_RequestControl component. The rate arguments should be those that are accepted by this component.

This component will terminate if it receives a shutdownMicroprocess or producerFinished message on its "control" inbox. The message will be passed on out of its "signal" outbox.

No producerFinished or shutdownMicroprocess type messages are sent by this component between one file and the next.

Development history

PromptedFileReader - developed as an alternative to ReadFileAdapter - prototyped in /Sketches/filereading/ReadFileAdapter.py


Kamaelia.File.Reading.FixedRateControlledReusableFileReader

prefab: FixedRateControlledReusableFileReader

FixedRateControlledReusableFileReader(readmode, rateargs) -> reusable file reader component

A file reading component that can be reused. Based on a carousel - send a filename to the "next" or "inbox" inboxes to start reading from that file.

Data is read at the specified rate.

Keyword arguments: - readmode = "bytes" or "lines" - rateargs = arguments for ByteRate_RequestControl component constructor

Kamaelia.File.Reading.PromptedFileReader

class PromptedFileReader(Axon.Component.component)

PromptedFileReader(filename[,readmode]) -> file reading component

Creates a file reader component. Reads N bytes/lines from the file when N is sent to its inbox.

Keyword arguments:

  • readmode -- "bytes" or "lines"

Inboxes

  • control : for shutdown signalling
  • inbox : requests to 'n' read bytes/lines

Outboxes

  • outbox : data output
  • signal : outputs 'producerFinished' after all data has been read

Methods defined here

__init__(self, filename[, readmode])

x.__init__(...) initializes x; see x.__class__.__doc__ for signature

closeDownComponent(self)

Closes the file handle

main(self)

Main loop

readNBytes(self, n)

readNBytes(n) -> string containing 'n' bytes read from the file.

"EOF" raised if the end of the file is reached and there is no data to return.

readNLines(self, n)

readNLines(n) -> string containing 'n' lines read from the file.

"EOF" raised if the end of the file is reached and there is no data to return.

shutdown(self)

Returns True if a shutdownMicroprocess message is received.

Also passes the message on out of the "signal" outbox.

Kamaelia.File.Reading.RateControlledFileReader

prefab: RateControlledFileReader

RateControlledFileReader(filename[,readmode][,**rateargs]) -> constant rate file reader

Creates a PromptedFileReader already linked to a ByteRate_RequestControl, to control the rate of file reading.

Keyword arguments:

  • readmode -- "bytes" or "lines"
  • rateargs -- arguments for ByteRate_RequestControl component constructor

Kamaelia.File.Reading.RateControlledReusableFileReader

prefab: RateControlledReusableFileReader

RateControlledReusableFileReader(readmode) -> rate controlled reusable file reader component.

A file reading component that can be reused. Based on a Carousel - send (filename, rateargs) to the "next" inbox to start reading from that file at the specified rate.

  • rateargs are the arguments for a ByteRate_RequestControl component.

Keyword arguments: - readmode = "bytes" or "lines"

Kamaelia.File.Reading.ReusableFileReader

prefab: ReusableFileReader

ReusableFileReader(readmode) -> reusable file reader component.

A file reading component that can be reused. Based on a Carousel - send a filename to the "next" inbox to start reading from that file.

Must be prompted by another component - send the number of bytes/lines to read to the "inbox" inbox.

Keyword arguments: - readmode = "bytes" or "lines"

Feedback

Got a problem with the documentation? Something unclear that could be clearer? Want to help improve it? Constructive criticism is very welcome - especially if you can suggest a better rewording!

Please leave you feedback here in reply to the documentation thread in the Kamaelia blog.

-- Automatic documentation generator, 19 Oct 2008 at 14:29:09 UTC/GMT

This is a page from the Kamaelia website. You can find the original here: