Joule Modules

File Reader: joule-file-reader

Read data in from a file or a named pipe. Optionally add timestamps to data (note, this should only be used for real time sources. The 8us difference in example below is the actual time taken to read the file line)

Example

SHELL

  $> cat /tmp/file
  3 4
  4 6
  $> joule-file-reader --timestamps=yes --file=/tmp/file
  1485274825371860 3.0 4.0
  1485274825371868 5.0 6.0

Inputs:

--None--

Outputs:

name	datatype	elements
output	float32	N auto detected from input source

Example Data Stream Config:

/etc/joule/data_stream_configs/output.conf

[Main]
name = File Data
path = /path/to/output
datatype = float32
keep = 1w

# [width] number of elements
[Element1]
name = Data Column 1

[Element2]
name = Data Column 2

#additional elements...

Example Module Config:

/etc/joule/module_configs/file_reader.conf

[Main]
name = Random Reader
exec_cmd = /opt/joule/bin/joule-file-reader

[Arguments]
timestamps = yes 
file = /absolute/file/path

[Outputs]
output = /path/to/output

Random Reader: joule-random-reader

This is a module that generates random numbers. Specify width (number of elements) and the rate in Hz.

Arguments:

Name	Description
width	number of elements
rate	data rate in Hz

Inputs:

--None--

Outputs:

name	datatype	elements
output	float32	N specified by width argument

Example Data Stream Configs

/etc/joule/data_stream_configs/output.conf

[Main]
name = Random Data
path = /path/to/output
datatype = float32
keep = 1w

# [width] number of elements
[Element1]
name = Random Set 1

[Element2]
name = Random Set 2

#additional elements...

Example Module Config:

/etc/joule/module_configs/random_reader.conf

[Main]
name = Random Reader
exec_cmd = /opt/joule/bin/joule-random-reader

[Arguments]
# number of elements
width = 4
# data rate in Hz
rate = 10 

[Outputs]
output = /path/to/output

Mean Filter: joule-mean-filter

Apply a moving average to the input stream with an adjustable window size

Arguments:

Name	Description
window	samples to average, must be odd

Inputs:

name	datatype	elements
input	any	N

Outputs:

name	datatype	elements
output	float32	N

Example Data Stream Configs:

inputoutput

/etc/joule/data_stream_configs/input.conf

[Main]
name = Raw Data
path = /path/to/input
datatype = int32
keep = 1w

[Element1]
name = Element 1

[Element2]
name = Element 2

#additional elements...

/etc/joule/data_stream_configs/output.conf

[Main]
name = Filtered Data
path = /path/to/output
datatype = float32
keep = 1w

#same number of elements as input
[Element1]
name = Element 1

[Element2]
name = Element 2

#additional elements...

Example Module Config:

/etc/joule/module_configs/mean_filter.conf

[Main]
name = Mean Filter
exec_cmd = /opt/joule/bin/joule-mean-filter

[Arguments]
# must be odd
window = 11 

[Inputs]
input = /path/to/input

[Outputs]
output = /path/to/output

Median Filter: joule-median-filter

Apply a windowed median filter to the input stream.

Name	Description
window	samples to average, must be odd

Inputs:

name	datatype	elements
input	any	N

Outputs:

name	datatype	elements
output	float32	N

Example Data Stream Configs:

inputoutput

/etc/joule/data_stream_configs/input.conf

[Main]
name = Raw Data
path = /path/to/input
datatype = int32
keep = 1w

[Element1]
name = Element 1

[Element2]
name = Element 2

#additional elements...

/etc/joule/data_stream_configs/output.conf

[Main]
name = Filtered Data
path = /path/to/output
datatype = float32
keep = 1w

#same number of elements as input
[Element1]
name = Element 1

[Element2]
name = Element 2

#additional elements...

Example Module Config:

/etc/joule/module_configs/mean_filter.conf

[Main]
name = Mean Filter
exec_cmd = /opt/joule/bin/joule-median-filter

[Arguments]
# must be odd
window = 11 

[Inputs]
input = /path/to/input

[Outputs]
output = /path/to/output

Merge Filter: joule-merge-filter

Combine multiple input streams into a single output stream. Specify a primary stream and one or more additional streams. The output will have timestamps from the primary stream and linearly interpolated data from all the additional streams. Uses scipy.interpolate.interp1d for resampling.

Arguments:

Name	Description
primary	name of the primary stream

Inputs:

name	datatype	elements
input1	any	N1
...	...	...
inputN	any	N_N

Outputs:

name	datatype	elements
output	float32	**N +...+ N_N **

Example Data Stream Configs:

input1input2output

/etc/joule/data_stream_configs/input1.conf

[Main]
name = Raw Data
path = /path/to/input1
datatype = int32
keep = 1w

[Element1]
name = Element 1

[Element2]
name = Element 2

/etc/joule/data_stream_configs/inputN.conf

[Main]
name = Raw Data
path = /path/to/inputN
datatype = int32
keep = 1w

[Element1]
name = Element 1

[Element2]
name = Element 2

/etc/joule/data_stream_configs/output.conf

[Main]
name = Merged
path = /path/to/output
datatype = float32
keep = 1w

# [width] number of elements of all input streams
[Element1]
name = Element 1 from input1

[Element2]
name = Element 2 from input1

[Element3]
name = Element 1 from input2

[Element4]
name = Element 2 from input2

Example Module Config:

/etc/joule/module_configs/merge_filter.conf

[Main]
name = Merge Filter
exec_cmd = /opt/joule/bin/joule-merge-filter

[Arguments]
primary = input1 

[Inputs]
input1 = /path/to/input1
#... additional inputs as desired
inputN = /path/to/inputN

[Outputs]
output = /path/to/output

Visualizer Filter: joule-visualizer-filter