OMC-048_FW_1.03B2732_H0003
Minor Bugfixes:
Fixed an issue causing the SDI and NMEA circuit power not to wake up from sleep while connected to USB.
Fixed an issue with some seametrics sensors sending no data if the second data request was issued to soon, 1 sec to 1.5 sec increase.
Corrected a changelog config using the wrong parameter name for “rxbuf”.
OMC-048_FW_1.03B2724_H0003
Added the control driver: Various requests were received to implement a “sensor driver” (See: Make Your Own Sensor Driver) without the need to specify a serial port. Also many customers wanted to implement their own custom code to trigger on a certain interval, without the need for data to be logged. These requests should now be answered due to the implementation of “control drivers”. See the Data Logger Application section on how to set up your own control driver.
Control drivers support the use of supply_ports, relay_ports, startup_time and cooldown_time. response_timeout can be used if the driver is conditional, the driver should return True when completed, otherwise it shall loopback until False is returned.
control_driver:
- id: control_driver_1
sample_interval: "* * * * *"
supply_port: 1
supply_port_always_on: False
response_timeout: 120
relay_port: 2
relay_port_always_on: False
startup_time: 5
cooldown_time: 1
Minor Bugfixes:
Corrected the readline bug in EXO and NEP drivers
Fixed a bug which caused abundant error/warning messages to impact the sample-time due to SD write actions.
OMC-048_FW_1.03B2712_H0003
Minor Bugfixes:
The modbus class formats were incorrectly defined, causing SIGNED_INT to be interpreted as SIGNED_SHORT.
Changed the order of initialization, this prevents the logger from powering sensors without an external power supply attatched during the parameter detection stage.
OMC-048_FW_1.03B2707_H0003
Modbus formats now support register and byte swap on all formats. Minor change in how to use: The formats DOUBLE_INV and FLOAT_INV have been removed, they are replaced by the new method:
class formats:
UNSIGNED_SHORT = 0b00000001 #WORD (1 register of 2 bytes) (AB)
SIGNED_SHORT = 0b00000010 #WORD (1 register of 2 bytes) (AB)
UNSIGNED_INT = 0b00000011 #DWORD (2 registers of 2 bytes) (AB CD)
SIGNED_INT = 0b00000100 #DWORD (2 registers of 2 bytes) (AB CD)
FLOAT = 0b00000101 #DWORD (2 registers of 2 bytes) (AB CD)
DOUBLE = 0b00000110 #QWORD (4 registers of 2 bytes) (AB CD EF GH)
#Swaps can be defined by adding (+) the desired swap formats:
REG_SWAP = 0b10000000 #Add REG_SWAP to have the registers swapped (AB CD) to (CD AB)
BYTE_SWAP = 0b01000000 #Add BYTE_SWAP to have the bytes swapped (AB CD) to (BA DC)
#A combination is possible: formats.DOUBLE + formats.REG_SWAP + formats.BYTE_SWAP (AB CD EF GH) to (DC BA HG FE)
The table can also be found in the modbus_reference_table file. If you update to this version from an older version, make sure to replace the old DOUBLE_INV and FLOAT_INV!
OMC-048_FW_1.03B2697_H0003
Added digital input state driver. By using input state it becomes possible to sample the state of the digital IO pins (51 & 52).
input_state:
- id: switch1
port: 1
state_invert: False
sample_interval: False
trigger_on_event: True
- id: switch2
port: 2
state_invert: False
sample_interval: "0,10,20,30,40,50 * * * *"
sample_interval: defines the cron interval on which to sample the input of the digital IO (state can be “0” of “1”). It’s an required parameter, which must be disabled by defining “False” when a sample interval is not applicable.
trigger_on_event: can be set “True” when a sample is to be taken as soon as the state changes. The change is detected immediately, only the sample is processed as soon as the datalogger wakes up from sleep. This means this setting is not optimal for fast changing states.
state_invert: is an optional parameter to invert the states. When set to “False” (default) = “0” is “open circuit”, “1” closed to GND. This is inverted when set to “True”
It is possible to use trigger_on_event in combination with sample_interval
Added additional functionality to the Data_serial_out driver:
Data_serial_out:
- id: data_serial
create_interval: "0 * * * *"
port: serial1
baudrate: 115200
mode: RS232
reference_table: MyTable
It is possible to enter an optional parameter named reference_table:. It can direct to a single or multiple tables, which must be defined in the file: serial_output_reference_tables.py. When set to None or removed entirely, the driver sends output according to the omc045 datalog format.
Whenever a table (reference_table: MyTable) or multiple tables (reference_table: MyTableA,MyTableB,MyTableC) are defined. The tables are used to convert the measured data to the defined format. This driver is designed to maintain the order in which the data should be presented. Whenever data was not available when the serial output sentence was transmitted, the value “nan” (not a number) is appended.
Example of a reference table (defined in serial_output_reference_tables.py)
MyTable = {
0: ('', ' ', '\r\n', checksum_list.MyTimestampFunction),
1 : (('Int. Coin cell voltage', 'V', 'IVBAT'), "{:.1f}"),
2 : (('Int. Temperature', 'C', 'ITEMP'), "{:.2f}"),
3 : (('Int. Humidity', 'rh', 'IHUM'), "{:.3f}"),
}
The repl output shows the following (both input_state as Data_serial_out are shown)
2024-01-03 12:49:52 [DATALOG_CUSTOM] INFO Initializing Serial message(s): ['MyTable']
2024-01-03 12:49:52 [DRIVER_MANAGER] INFO input_state detected: switch1
2024-01-03 12:49:52 [DRIVER_MANAGER] INFO input_state detected: switch2
2024-01-03 12:50:00 [DATALOG_CUSTOM] INFO Serial message: MyTable is ready for transmission
2024-01-03 12:50:00 [SERIAL_OUT] INFO Starting serial transmission
2024-01-03 12:50:00 [DRIVER_MANAGER] INFO Sensor [switch1] data successfully obtained
Sensor data: ('Input State', '', 'DI_switch1') 0
2024-01-03 12:50:00 [SERIAL_OUT] INFO Transmission over serial completed
2024-01-03 12:50:00 [DATALOG_CUSTOM] INFO Serial message: LOMANS is ready for transmission
The output over the defined serial port shows:
# 24/01/03 12:50:00 3.0 22.05 43.467
# 24/01/03 12:51:00 3.1 22.15 43.867
# 24/01/03 12:52:00 3.2 22.15 43.867
Added Spectrolyser driver
Added the spectrolyser modbus driver:
modbus_spectrolyser:
- id: spect
port: serial2
baudrate: 38400
mode: RS485
bits: 8
parity: None
stop: 2
rxbuf: 256
protocol: RTU
relay_port: 1
relay_port_always_on: True
startup_time: 6
sample_interval: "35 8,18,28,38,48,58 * * *"
measurement_wait_time: 25
Updated modbus reference tables.
Fixed a bug not supporting 8 bits in serial parity.
Navilock GPS driver now only rejects valid data when the GPS Fix flag is zero (invalid)
OMC-048_FW_1.03B2683_H0003
Bugfix on Data_serial_out and generic_serial_output which caused a freeze when rx_buf flooded.
OMC-048_FW_1.03B2677_H0003
D-record output over serial bugfix which caused the checksum to be too short when under 4 digits total.
OMC-048_FW_1.03B2653_H0003
Added relay_port: Use relay 1 or 2 just like the power_supply ports!
my_driver:
- id: my_driver_example
port: serial1
baudrate: 1200
bits: 7
parity: 0
reference_table: PT12,PTX
sample_interval: "0 * * * *"
wipe_interval: "0 0,10,20,30,40,50 * * *"
supply_port: 1
supply_port_always_on: True
relay_port: 2
relay_port_always_on: False
startup_time: 3
cooldown_time: 3
The relay also use cooldown and startup times, and can be enabled permanently using relay_port_always_on. It is possible to use a power supply in combination with a relay, but not more then one of each type! The relays shall return to the off state (the state as presented on the front sticker of the omc-048) when the power drops below critical and the logger shuts down.
Added OPUS driver: Added a modbus based driver for the TrIOS OPUS sensor, including a generic table showing all the data registers. Special to the OPUS driver is the measurement_wait_time: 25 in config, this refers to the time between the ‘start measurement’ commmand and ‘read data’ command. This time is dependant on the configuration of the OPUS. If the time is to narrow the sensor shall respond in modbus errors, indicating it is not ready. Use this feedback to verify the time the sensor needs while setting up the driver. Do not forget to increase the response_timeout: 120 in case the total measurement time exceeds the default of 120 seconds.
modbus_opus:
- id: modbus_opus
sample_interval: "0 * * * *"
port: serial1
baudrate: 9600
rxbuf: 1024
mode: RS485
protocol: RTU
reference_table: TriOS_OPUS
measurement_wait_time: 25
register_print: True
response_timeout: 120
D-record output over serial implemented. This output driver is configured like the original Data file driver. The “create_interval” and the serial port (with the optional usual port settings) must be defined for correct operation. The message is simular to the OMC-045 data log format (as used in the normal log files), only a header (OMC-045_<system_id>_<logger_serial_number>) and checksum (CRC16) are appended. All data obtained from sensors is appended to the serial output message, so make sure the create_interval and transmission baudrates are adequate!
Note that only one “Data_serial_out” object can be created at a time, the serial port is free to choose.
The message is according to the following format: <0x02>(Start of Text)<message><0x03>(End of Text)<CR><LF>
config.txt:
# ----Output-settings---- #
Data_serial_out:
- id: data_serial
create_interval: "0,10,20,30,40,50 * * * *"
port: serial1
baudrate: 115200
mode: RS232
output over serial1:
OMC-045_Test_Logger_048000201
T;25;26;27;28;
Timestamp;Int. Temperature;Int. Humidity;Int. Coin cell voltage;Supply voltage;
yy/mm/dd hh:mm:ss;C;rh;V;V;
Time;ITEMP;IHUM;IVBAT;VSUP;
D;23/05/16 07:25:27;25;23.4699;0;26;47.96674;0;27;2.901099;0;28;24.09611;0;
D;23/05/16 07:25:28;25;23.4699;0;26;47.94384;0;27;2.933333;0;28;24.13456;0;
D;23/05/16 07:25:29;25;23.45655;0;26;47.87518;0;27;2.897875;0;28;24.13456;0;
991D
repl:
2023-05-16 07:29:50 [DATALOG045] INFO Closed file data/OMC-045_Test_Logger_tmp_230516_072940.txt
2023-05-16 07:29:50 [DATALOG045] INFO Started logging in 'data/OMC-045_Test_Logger_tmp_230516_072950.txt'
2023-05-16 07:29:50 [SERIAL_OUT] INFO Starting serial transmission of file: 'data/OMC-045_Test_Logger_tmp_230516_072940.txt'
2023-05-16 07:29:50 [DRIVER_MANAGER] INFO Sensor [board_sensors] data successfully obtained
Sensor data: ('Int. Temperature', 'C', 'ITEMP') 23.63813
Sensor data: ('Int. Humidity', 'rh', 'IHUM') 47.40826
Sensor data: ('Int. Coin cell voltage', 'V', 'IVBAT') 2.891428
Sensor data: ('Supply voltage', 'V', 'VSUP') 24.22107
2023-05-16 07:29:50 [SERIAL_OUT] INFO Transmission over serial completed, removed file:'data/OMC-045_Test_Logger_tmp_230516_072940.txt'
Fixed a bug that caused the 5V power_supply to shutdown during sleep.
The detect parameter step now properly waits the “startup_time” to match the behaviour of a normal measurement cycle
Added some extra syslog info on powering and shutting down the supply ports and relays.
Updated modbus reference tables.
The modbus_seametrics and modbus_opus driver files are non-frozen these examples can be used to tweak your own mobbus sensor
OMC-048_FW_1.03B2615_H0003
Updated modbus and serial reference tables.
Added 6 digit register address support for modbus.
Added modbus_seametrics, based on the modbus driver. The driver is identical to modbus except the data request is handled twice with a delay of 1 second in between. The first response is discarded, as this was only used to initiate a new measurement for the seametrics sensor.
modbus_seametrics:
- id: modbus_1
protocol: RTU
register_print: True
sample_interval: "0 * * * *"
port: serial1
mode: RS485
rxbuf: 1024
baudrate: 9600
reference_table: modbus_table_1,modbus_table_2
OMC-048_FW_1.03B2600_H0003
Modbus RTU driver added. Via the new generic modbus driver you can configure your own modbus sensor.
modbus:
- id: modbus_1
protocol: RTU
register_print: True
sample_interval: "0 * * * *"
port: serial1
mode: RS485
rxbuf: 1024
baudrate: 9600
reference_table: modbus_table_1,modbus_table_2
Via the config the table(s) are referenced via “reference_table”. The raw registers can be shown via the config setting via “register_print”, this can help by setting up the modbus tables. Registers are printed as unsigned short notations and can be converted to the prefered notation via the reference tables. How to use the reference tables is explained in the file: “modbus_reference_tables.py”.
The protocol currently only supports functionality to read registers, also only RTU is supported. It can be used on any port type, and via RS485 as a bus.
OMC-048_FW_1.03B2559_H0003
New remote redirect functionality added. It’s now possible to redirect a serial port via the remote server. First the serial port object should be created and initialized via the repl. The created port should be given as an argument to the remote.redirect(“serial_object”) method.
Example how it could be entered into the repl:
from remote_logger_api import RemoteLoggerApi; remote = RemoteLoggerApi('tcp'); import omc048; x = omc048.serial(3); x.init(57600, x.RS485); remote.redirect(x)
All the ASCII characters and CR-LF will be printed in the terminal as readable, all the other characters will be printed in hexadecimal format ‘xFC’.
OMC-048_FW_1.03B2551_H0003
Minor patches / fixes:
The generic serial input now checks if the given data parameters do not exceed the table parameters.
The generic serial input now properly supports multi character separators.
The generic serial input now properly checks for a start character/string, if not present it is not detected or a timeout shall occcur.
OMC-048_FW_1.03B2544_H0003
Minor patches / fixes:
Fixed a bug which caused usb not to show when the backup battery was disconnected.
Fixed a bug which caused the usb not to reconnect to mass storage if the reset button was pushed during self-test.
Added cron_offset: to Connection_settings:, this makes it posible to also offset the cellular data items.
Replaced cellular_diagnostics: True setting from Modem: to Connection_settings:. This is the more logical place to set the setting. It still works under Modem: for backwards compatiility.
Initialisation time of modem changed to 20sec due too new version of modem-FW.
Fixed error on empty webserver password in config, now initializes empty string when left empty.
When config is corrupt/invalid the logger should start the recovery, but not when the USB is connected to the logger, the logger now shows the message “Recovery not starting, due to active USB connection!”.
Fixed a bug that disabled the usb chip in storage mode when powered by only usb.
Fixed a bug caused by a timout on the context activation by the at-parser. Modem kept status busy.
OMC-048_FW_1.03B2519_H0003
Minor patches / fixes:
Navilock sensor driver fix on the hemisphere N-S and S-W, now properly changing the lat en lon to positive and negative.
A incorrect recovery.txt on user- or device flash could cause the logger to power cycle loop, now fixed.
Disabling th usb chip via the repl could lock up the device for REPL entering, now fixed. On power cycle the usb chip is now always enabled.
Updated and added the web-manual.
recovery_reset no longer imprints a dummy template but deletes the content instead.
OMC-048_FW_1.03B2495_H0003
New rain pulse driver added to the collection of the drivers. data acquisition is used. Both digital inputs can be used. The interrupt on the inputs will be triggered by falling edge. This means the rain gauge should switch/connect the ground to the digital input.
rain_pulse:
- id: rain_pulse0
port: 1
mm_per_pulse: 4
reset_interval: "0 0 * * *"
sample_interval: "0,10,20,30,40,50 * * * *"
Minor patches / fixes:
Cellular information wasn’t showing on the B2C server caused by early reset of data in the connection manager, now fixed.
The SSL selftest failed when connecting to the B2C server during the test, now fixed.
FW version now updated to the 1.X range.
The EXO driver now detects when no wiper is attached to the EXO, if no wiper attached the wiper timeout will be skipped.
Errors occurred when a CRON interval was configured with long periods in between(week), now fixed.
Time format of the selftest and the SSL-transfer changed to meet the UTC standard (yyyy-MM-ddTHH:mm:ss.fffZ)
OMC-048_FW_0.03B2434_H0003
Introduction of SSL data transfer
It is now possible to transfer log files via SSL to a server. The introduction of this feature required a major change in the config.txt settings regarding the external connection settings.
The original ‘ftp’ driver has been removed and is implemented in the following driver:
# ---- External connection settings ---- #
Connection_settings:
webserver_access: True
data_protocol: None
webserver_url: blue2cast.com
webserver_port: 5000
webserver_password: <OBSERVATOR GENERATED HASH>
ftp_url: ftp.omc-data-online.com
ftp_port: 21
ftp_username: OMC-test
ftp_password: omc-test
utc_time_sync: False
utc_time_server: europe.pool.ntp.org
utc_time_server_port: 123
transmit_interval: "0 0,5,10,15,20,25,30,35,40,45,50,55 * * *"
# ----Modem-Settings---- #
Modem:
cellular_diagnostics: False
It is still possible to ftp, only it is now selected via the ‘data protocol’ selection (FTP or SSL). The user can now select which ‘external connection services’ are required. Possible choices are:
FTP data transmission or SSL data transmission
NTP timeserver sync
Webserver access.
Note that these various services often do require sub-settings as given in the example above. The cellular_diagnostics is now a modem setting.
Timestamp correction
Under normal circumstances the timestamp of each data sample is stamped on the time it was obtained from the sensor. The gives the most accurate measurement time, however some users preferred to have it stamped on the time the measurement it was scheduled (at the given CRON intervals). It is now possible to have the log data stamped on given cron intervals instead of the actual time of measurement. The setting can be activated by “cron_offset: HH:MM:SS” and can be set for each individual sensor/driver. By entering the offset of ’00:00:00’ the most recent passed driver sample interval is used. The offset given is simply added to this time notation. If cron_offset is not provided the original time of data acquisition is used.
- id: simpy1
sample_interval: "0,10,20,30,40,50 * * * *"
average_interval: "0 * * * *"
cron_offset: 00:00:01
port: serial1
Minor patches / fixes:
Transmission of large files over remote would end up in a timeout reset. The inactivity timeout is now set to 10 minutes and is refreshed upon file down or uploads
The generic_serial_output.py driver also transmitted unobtained data over the serial port for every parameter which was found in the table. Also non float values caused an error.
A combination of averaging and serial out caused incorrect behavior by sending serial out data only on measurement intervals but not averaged intervals. Now data is only transmitted on averaged intervals as it would with normal D-record (normal logs)
A rename on ‘data_interval’ to ‘average_interval’. Also the ‘checksum’ function in the generic tables was renamed to ‘reformat_output’.
The selftest result is now parsed to JSON format for future compatibility.
OMC-048_FW_0.03B2271_H0003
Remote server support
The remote support server is now online! Currently only available for Observator service department, but soon available for customers. The config.txt can now be set-up to periodically (on start-up and ftp-interval) contact the remote server.
# ----FTP-Settings---- #
Ftp:
- id: data_ftp
url: ftp.omc-data-online.com
port: 21
username: OMC-test
password: omc-test
transmit_interval: "5 * * * *"
utc_time_sync: True
webserver_remote_access: True
webserver_url: blue2cast.com
webserver_port: 5000
webserver_password: 96e61649950d5c16d3cee4703da0956671f84c2998ec43ae487fa90d8e26f668
The settings above shall enable the datalogger to contact the server to provide notifications to the user while in operation. Also, the logger can be ‘awaited’ by the server so various tasks can be performed. Remote features include: Repl control, firmware update, editing the config.txt, syslog view while in operation, file upload and download e.t.c. In other words, the logger can fully be controlled via the remote server. The server connects via SSL with additional handshake to give secure access. The password which is given in the config must be generated by Observator for safety reasons.
Recovery mode
In addition to the remote server support a ‘recovery mode’ is introduced to prevent the datalogger from becoming inaccessible due to scripting, missing drivers, or configurational errors. To use the recovery feature, the ‘recovery.txt’ file is added to the logger’s flash drive. This file is like the config.txt but it is set up to only access the remote server, so the logger can be ‘recovered’ and restored to a working condition.
# ----System---- #
Omc048:
usb_mode: debug
system_id: RECOVERY
# ----Modem-Settings---- #
Modem:
id: onboard_modem
port: modem
sim_username:
sim_password:
apn: advancedinternet
network_technology: GSM + LTE
# ----FTP-Settings---- #
Ftp:
- id: data_ftp
webserver_remote_access: True
webserver_url: blue2cast.com
webserver_port: 5000
webserver_password: 96e61649950d5c16d3cee4703da0956671f84c2998ec43ae487fa90d8e26f668
transmit_interval: "0 * * * *"
Recovery mode to flash
Additionally, it is possible to have the recovery.txt file to be stored on the internal flash device. In case the original recovery.txt is lost or deleted the recovery mode shall fall back to the latest internally stored recovery.txt. This process requires a functional recovery.txt on the loggers flash root (with correct password settings and sim provider settings). See the omc048 Webserver set-up guide on how to properly set up the device for recovery.
Generic serial out driver
Added a generic serial output driver, via “generic_serial_output_reference_tables.py”. This driver replaces the “OMC2900_reference_table_template.py” (omc2900 serial output driver). The driver can be set-up via the reference table to convert data items into an output string. Multiple reference tables can be created, and even a checksum can be added! Please see the description on how-to-use in the file “generic_serial_output_reference_tables.py”.
generic_serial_output:
- id: output_1
port: serial1
reference_table: OMC2900
baudrate: 9600
mode: RS232
Data averaging
All data logged by the logger can now be averaged over a given data_interval. The sensor will still sample data at the configured sample interval but will process and save the data to the log_file at the data_interval. The expected samples is the minimum amount of samples needed to set the “data-flag” to valid. If less than the minimum_required_samples are collected the “data-flag” will be set to invalid, but the data will still be processed.
#----Sensor-Settings---- #
Onboard:
- id: board_sensors0
sample_interval: "0,20,40 * * * *"
data_interval: "30 * * * *"
minimum_required_samples: 30
Power monitor settings (*hardware #048000201 and upwards)
With the new hardware revision it’s possible to monitor the power supply. New settings are added so a hysteresis on the input supply can be configured. When the voltage of the supply drops under the hysteresis_low, the logger will close all log-files and go to sleep till the voltage rises above hysteresis_high. In case of a power failure the logger just closes the files and will restart when the sufficient power is reapplied.
# ----Power Monitor---- #
Input_power_monitor:
- id: power_mon
hysteresis_high: 12
hysteresis_low: 11.5
sample_interval: "0,20,40 * * * *"
Minor patches / fixes:
SDI-12 driver now checks if the configured reference table is too long when detecting the parameters, if the table is too long, the sensor will be removed from bus to prevent errors. The driver now gives a timeout when no data is collected.
Self-test now checks the drivers on configured reference tables and shows if these sensors are connected per table.
Remote_logger_api now checks if there’s a “NO CARRIERrn” in the data block when it runs through a timeout, so the logger can performs a reset when it lost connection with the server during a file upload from the server.
Fix on the generic serial driver, the stop character only parsed a single character if multiple were given in the table.
Fix on generic sdi driver, not properly separating in some cases.
Rename on the “generic_serial_reference_tables.py”, it is now called: “generic_serial_input_reference_tables.py”
Fix on the config parser to support ‘:’ in strings
OMC-048_FW_0.03B1900_H0003
Generic SDI12 driver
The SDI12 protocol is now supported by the generic_sdi12.py driver. The driver is based on the concurrent commands, so all sensor measurements can be started at the same time/one after another.
# ----Sensor-Settings---- #
Generic_sdi12:
- id: gen0
port: serial1
baudrate: 1200
bits: 7
parity: 0
reference_table: PT12,PTX
sample_interval: "0 * * * *"
wipe_interval: "0 0,10,20,30,40,50 * * *"
supply_port: 1
supply_port_always_on: True
The driver uses a reference table with the same layout as our other reference tables. Further explanation can be found in the sdi12_reference_table_template.py.
OMC2900
The OMC2900 driver now has a separate reference table called “OMC2900_reference_table_template.py”. The handling of the table hasn’t changed.
Iridium
The Iridium driver now has a separate reference table called “iridium_reference_table_template.py”. The handling of the table hasn’t changed.
Generic NMEA
The name of the configured sentences in the config is changed from “nmea_sentence” to “reference_table”. The name “generic_nmea_reference_table.py” changed to “nmea_reference_table.py”, the handling hasn’t changed.
# ----Sensor-Settings---- #
generic_nmea:
- id: nmea_gen0
port: serial1
reference_table: VTG,GGA,RMC
sample_interval: "0,10,20,30,40,50 * * * *"
response_timeout: 60
Generic serial
The name “generic_serial_reference_tables.py” changed to “generic_serial_reference_tables_template.py”, the handling hasn’t changed.
Onboard
On the new hardware (44613-03) it’s possible to log the power supply voltage. The parameter and data will be appended to the onboard sensor-data.
Hardware modifications applicable to PCB 44613-03
The DC-supply-port-4 now delivers 5VDC voltage instead of a 12VDC.
The analog output reference voltage now is 5VDC.
The power supply voltage now can be monitored by the logger.
The USB-supply voltage now can be monitored by the logger.
Minor patches / fixes:
Driver wasn’t able to send via RS485 caused by an micropython update. Fixed by switching back to previous send function.
Fix on removing items with log_parameters, the cellular data can now be selected by user.
The generic serial driver now accepts strings of more than one character to be used as a start character.
On request the onboard sensor measurements now contain Int. in the parameter name so they can be easily distinguished from external sensors.
OMC-048_FW_0.01B1598_H0002
Minor patches / fixes:
‘Log_parameters’ was not excluded which caused an exception as it was interpreted as a serial driver.
The self-test files and config files created with the slf-test routine are now renamed because of issues with OMC DOL.
OMC-045_system_id_serialnr_YYMMDD_HHMMSS-config.txt
OMC-045_system_id_serialnr_YYMMDD_HHMMSS-test.txt
The pyb.uart_repl() can now be set to a serial port class as well as a uart class.
OMC-048_FW_0.01B1595_H0002
Config.txt reposition to flash
From now on the config.txt needs to be placed on the flash of the microcontroller. If the flash on the OMC-048 already is defined, the mass-storage folder that presents itself on usb connection will be called “PYBFLASH”. The “PYBFLASH” folder could contain files, such as: readme, main.py, boot.py etc. All the files on the PYBFLASH can be removed and the config.txt should be placed on the root of the flash device. In case of a more recent logger the flash will be called “OMC048FLASH”.
USB mode adaptions
The config gives you an option in the system settings to switch between two different USB-modes, called the “storage” and “repl” -mode. The config file is now stored on the internal flash of the microcontroller, so regardless of the mode, it will always be possible to make changes to the config.txt file. The third mode which can be used is the “debug” mode. The debug-mode is not recommended for normal operation, cause of the possibility of files getting corrupted. “storage” mode When configured in storage-mode the OMC-048 will stop its routine when the USB-cable is plugged in. With the USB-cable connected it is possible to copy/transfer the files from the logger to the host and access the files. When the USB-cable is removed the logger will restart its application as defined by the configuration. Be aware it is not possible to start the logger application with USB connected, when in storage-mode. “repl” mode When configured in repl-mode the OMC-048 will continue its routine when a USB-cable is plugged in. It is possible to open a terminal emulator to start/stop/watch the (real-time) routine of the OMC-048. When closing the terminal window or disconnecting the USB, the logger will continue its routine without restarting the application. In case the logger was halted via the REPL (Ctrl-c) and the USB cable was removed, the application is restarted automatically. Be aware that there’s no possibility to reach the data- and logfiles in repl-mode. “debug” mode When configured in debug-mode the OMC-048 will give access to the mass-storage and can be started/stopped/watched via a terminal emulator with the REPL, as long as the USB-cable is plugged. Be aware the files can become corrupted, cause the SD card access is now shared between the host device and the logger application. The text-box below shows an example of a config with the usb_mode setting included.
# ----System---- #
Omc048:
usb_mode: storage
system_id: OMC048_Tst
application: example
file_log_level: info
repl_log_level: info
utc_time_offset_hours: +1
utc_time_offset_minutes: 0
sensor_data_print: True
The usb_mode is a required setting! The application cannot start unless this setting is defined in the config.txt! USB wake removed from configuration
This functionality is now included in the USB-mode setting. There is no need and possibility to do this manually anymore. The usb connection is checked every 10 seconds so it might take some time for the omc048 device to appear on screen.
Self-test on startup
The self_test adds the possibility to check if all sensors configured in config.txt are connected and can send their sensor data to the OMC-048. If FTP is configured the self_test driver will test its connection by transmitting the config.txt. If all parameters of all configured sensors are collected and the FTP-task (if configured) passed, the driver let the user know the self_test passed by printing it in the terminal emulator and transmit the self-test results via FTP to the configured server.
# ----System---- #
Omc048:
usb_mode: MODE
system_id: OMC048_Tst
application: example
file_log_level: info
repl_log_level: info
utc_time_offset_hours: +1
utc_time_offset_minutes: 0
sensor_data_print: True
self_test: True
The self_test setting is enabled by default!
The files are transmitted to the same ftp adres as the log/data files. The config file is named: OMC-045_system_id_serialnr_YYMMDD_HHMMSS_config.txt This is an exact copy of the original config.txt on the data loggers flash storage.
The self-test file is named: OMC-045_system_id_serialnr_YYMMDD_HHMMSS_test.txt The self-test files are stored in the data/self-test folder on the SD storage of the data logger.
Eureka Manta driver
The manta protocol is supported via the manta driver. The wipe interval needs to be set on the same occurrence of a data sample or it shall be delayed until the next sample interval cron.
manta:
- id: MAN1
port: serial1
sample_interval: "0 * * * *"
wipe_interval: "0 0,10,20,30,40,50 * * *"
supply_port: 1
baudrate: 19200
supply_port_always_on: True
Nortek Aquadopp driver
The Aquadopp binary output is supported via the Aquadopp driver. Make sure you set the response timeout a bit longer than the profiling time of the Aquadopp.
# ----Sensor-Settings---- #
nortek_cp:
- id: aqdp1
port: 3
baudrate: 19200
sample_interval: "15 0,30 * * *"
response_timeout: 900
Generic NMEA driver
There now is a generic NMEA driver, which is capable of decoding any NMEA sentence, as long as the sentence is defined in “generic_nmea_reference_tables.py”. Specific instructions how a NMEA-sentence is added can be found in the “generic_nmea_reference_tables.py” file as well. It’s possible to decode multiple NMEA-sentences by defining the “to be decoded” sentences in the “nmea_sentence:” setting. Be aware if one of multiple sentences is not recognized or decoded, the driver will give a sensor timeout. Make sure there is no whitespace between the sentences.
# ----Sensor-Settings---- #
generic_nmea:
- id: gn1
port: 3
baudrate: 9600
sample_interval: "0 * * * *"
response_timeout: 5
supply_port: 2
supply_port_always_on: True
nmea_sentence: RMC,GGA
Iridium timeout handling
The iridium driver can now be altered to add sensor data as blank fields on timeout occurrences. If the setting is disabled (default) no data shall be appended to the iridium message.
# ----Iridium-Settings---- #
Iridium:
- id: data_iridium
system_id: IUQ
transmit_interval: "55 0,15,30,45 * * *"
add_timeout_field: True
Cellular diagnostics
Via the FTP settings it is now possible to add cellular diagnostics data to the log. The cellular diagnostics consist of signal strength, provider, roaming and network technology. If enabled the data is appended to the log as it would with normal sensor data according to the transmit interval cron given to the FTP driver. Cellular diagnostics is disabled by default.
# ----FTP-Settings---- #
Ftp:
- id: data_ftp
url: ftp.omc-data-online.com
port: 21
username: OMC-test
password: omc-test
transmit_interval: "20 * * * *"
utc_time_sync: True
utc_time_server: nl.pool.ntp.org
utc_time_server_port: 123
cellular_diagnostics: True
Minor patches / fixes:
Improved the use of memory allocation in various functions
Serial drivers can now be added to the modules folder without the need for the firmware to be updated.
Exo driver wiper implementation changed from: ‘wipe_after_numer_of_samples:’ to ‘wipe interval:’
OMC-048_FW_0.01B1463_H0002
Added ‘startup_time’ to config
It is now possible to define the ‘startup_time’ in the config. This gives more control over each specific sensor using the same driver. The startup time defines the time in seconds for the power supply to be enabled prior to a measurement. It allows a sensor to ‘stabilize’ before the data is requested. The setting is optional and only relevant in case ‘supply_port_always_on: False’.
The example config below shows an analog port which has a 10 second “startup_time” after the power had been applied:
# ----Sensor-Settings---- #
Analog_voltage:
- id: analog_voltage1
port: 1
supply_port: 2
supply_port_always_on: False
startup_time: 10
sample_interval: "0,10,20,30,40,50 * * * *"
min_in: 0
max_in: 24000
min_out: 0
max_out: 24
log_name: Power supply
log_unit: V
log_tag: VOLT1
The startup_time setting can also be used to overwrite defaults. For example the Manta driver maintains a default startup time of 20 seconds, the EXO maintains a default op 30 seconds. In case the config contains a new startup_time, the default for this specific sensor is overwritten.
Minor patches / fixes:
The ysi-6-series parameter table is updated
Fixed the problem of memory consumption while trying to load a decent size config.
OMC-048_FW_0.01B1406
Minor patches / fixes:
The gmx501 driver now supports GPS LAT and LON view on OMC-DOL. As with the navilock GPS the device needs to obtain a valid gps signal to append the data. Make sure the “response_timeout” is set adequately.
The navilock gps now appends tag_id data to all data items except for LAT/LON. A bug is fixed which caused the parameters not to be detected in case of a RMC message missing time and date data.
The omc048 should only be fitted with one GPS sensor, if multiple gps sensors are used the “Log_parameters” setting should be used to prevent dual ‘LAT and LON’ tags from the sensors.
The omc DOL data status codes now set and reset properly for all sensors. Data which is appended by a sensor which contains “” (empty data field) or “None” is now considered invalid and the invalid tag is set to this item.
The creation of a new data record was according to the current time for each individual data item. This caused the data from a single sensor to be split up over two records. The data from a single sensor is now appended on a single record, which also improves system speed.
The iridium driver now appends ‘empty’ (tag;data) data to the message in case of a timeout. This way it is possible to maintain a fixed position message if the scheduling is done correctly. This is at the cost of data reduction.
OMC-048_FW_0.01B1385
Minor patches / fixes:
The Iridium driver serial port initialisation changed to no flow control as default.
OMC-048_FW_0.01B1370
Added generic serial driver
Via the generic_reference_tables.py it is now possible to easily create a standard driver for most common serial drivers according to the “start, separator, stop” format. A How-To-Use is provided in the generic_reference_tables.py file. The gmx501 and gmx500 drivers are provided for use and as an example on how to set-up a generic driver. Additional generic tables shall be added in the future.
# ----Sensor-Settings---- #
generic_serial:
- id: gen0
port: serial1
reference_table: gmx500
sample_interval: "0,10,20,30,40,50 * * * *"
response_timeout: 60
Added Gil gmx501 serial driver
The gmx501 sensor driver is added. It is built on the concept of the generic driver, but it required additional functionality for the checksum calculation and the date & time update for the logger. The sensor is configured as any other serial sensor with the additional setting to update the omc048 utc time automatically.
# ----Sensor-Settings---- #
gmx_501:
- id: gen0
port: serial1
sample_interval: "* * * * *"
response_timeout: 60
utc_time_sync: True
supply_port_always_on: True
Minor patches / fixes:
The EXO table is updated with unique tag names.
The onboard sensor measurements routine is optimised in speed and to prevent the task from blocking other tasks in case of multiple tasks at the same interval.
OMC-048_FW_0.01B1320
Added iridium modem
It is possible to transmit iridium modem data via the Iridium Edge modem. An iridium modem is configured via the config menu as well as the iridium.py file.
At first the hardware is defined; a serial port is chosen and an optional supply port for switchable power. The baudrate does not need to be configured, this shall be set automatically to 19k2.
# ----Modem-Settings---- #
Iridium_modem:
- id: iridium_modem
port: serial4
supply_port: 4
Iridium settings must be defined in which ‘system_id’ can be changed to preserve data for the iridium message. If ‘system_id’ is not defined the ‘system_id’ name of the omc048 settings shall be used as iridium message header. The ‘transmit_interval’ is used to schedule transmission attempts.
# ----Iridium-Settings---- #
Iridium:
- id: data_iridium
system_id: ZZ
transmit_interval: "0 0 * * *"
Iridium messages are stored in the data folder just like normal log files. The files are redirected to the data/iridium folder once transmitted or in case of failed attempts. The ‘Iridium_file’ task defines the interval in which a new iridium file is created (which shall be picked up for transmission by the Iridium driver, given above)
# ----Data-log-settings---- #
Iridium_file:
- id: iridium
create_interval: "55 59 * * *"
Sensors - unlike normal log outputs - only add data to the iridium message (file) if defined by the dictionary in iridium.py. By setting the ‘sensor_data_print: True’ in the system settings, all measured sensor data is printed in the REPL. This data should be used to setup the dictionary of the iridium.py file.
self.conversion_dict = {
#Sensor TAG #Iridium TAG #Nr of decimals #Char clip (includes +/-/,)
'VBAT' : ('V', 1, 4, ),
'TEMP' : ('T', 2, 5, ),
'HUM' : ('H', 3, 8, ),
'LAT' : ('L', None, None, ),
'LON' : ('l', None, None, ),
# 'user defined' : (None, None, None, ),
}
The original #Sensor TAG is used for identification, if present the converted data is appended to the iridium message. Also, the number of decimals can be set as well as a clip on the number of characters. If the data should be ‘as-is’ please define ‘None’. In case values exceed the range of the number of characters, the shall be clipped to the maximum or minimum depending on the value.
Below some examples of valid ranges, versus number of decimals:
-999 to 9999 (0 decimals, clip to 4)
-99.9 to 999.9 (1 decimals, clip to 5)
-9.99 to 99.99 (2 decimals, clip to 5)
If ‘sensor_data_print: True’ is active the iridium message is printed prior to transmission. A message contains the ‘system_id’ at start, then a data/timestamp, after which the data appended by the sensors (via the dictionary filter). This should be used to validate if the conversion is made accordingly.
Iridium message example: *ZZ;210222143455;T;23.77;H;44;V;3.1;L;51.89074;l;4.161869
WARNING: If sensors transmit data at a higher rate then the iridium file is created, identical data fields shall be appended to the message.
WARNING: If iridium files are created at a higher rate then the iridium transmission rate, only one iridium file shall be transmitted, the others repositioned to the data/iridium folder without being transmitted. As the file to transmit is not prioritized the user should make sure only one file is present for transmission.
WARNING: only 338 characters can be transmitted via iridium, once every three minutes. If the iridium file contains a message which exceeds the length, it is not transmitted! Is shall be relocated to the data/iridium folder.
WARNING: Iridium can be used parallel to the normal data log (ftp) routine. Beware that high output rate sensors are not picked up by the iridium dictionary as the will fill up the iridium message quickly!
WARNING: The iridium receiver is a line-of-sight device and should be positioned in an open area without obstacles. In case transmission attempts fail the date is not re-sent on a later interval as is done with FTP data.
Improved modem routine
Some diagnostics are added to the modem routine to provide more information in case of signal reception/sim/settings issues. The cellular provider, signal strength and used technology are presented via a syslog message on each transmission attempt. Also, various ‘abnormal’ modem responses are translated to syslog messages to provide better feedback at which stage the transmission fails.
2021-03-08 15:10:30 [FTP] INFO Onboard modem powered, power on delay 10 sec
2021-03-08 15:10:30 [SCHEDULER] INFO Sleep for 9.8 seconds
2021-03-08 15:10:40 [MODEM] INFO Registered to network, roaming
2021-03-08 15:10:40 [MODEM] INFO Technology: "GSM", provider: "NL KPN", signal strength: "-54 dBm"
2021-03-08 15:10:59 [FTP] INFO Starting transfer
2021-03-08 15:11:02 [FTP] INFO Transfer of data/OMC-045_Scrilo_049000000_210308_150243.txt completed
2021-03-08 15:11:02 [FTP] INFO Untransmitted data file detected: data/OMC-045_Scrilo_049000000_210308_150401.txt
2021-03-08 15:11:04 [FTP] INFO Transfer of data/OMC-045_Scrilo_049000000_210308_150401.txt completed
2021-03-08 15:11:04 [FTP] INFO Untransmitted data file detected: data/OMC-045_Scrilo_049000000_210308_150601.txt
2021-03-08 15:11:07 [FTP] INFO Transfer of data/OMC-045_Scrilo_049000000_210308_150601.txt completed
2021-03-08 15:11:07 [FTP] INFO Untransmitted data file detected: data/OMC-045_Scrilo_049000000_210308_150801.txt
2021-03-08 15:11:10 [FTP] INFO Transfer of data/OMC-045_Scrilo_049000000_210308_150801.txt completed
2021-03-08 15:11:10 [NTP] INFO NTP server check initiated
2021-03-08 15:11:11 [NTP] INFO Logger time deviation to NTP source > 2 second
2021-03-08 15:11:15 [NTP] INFO Logger time synchronised to NTP source
2021-03-08 15:11:15 [FTP] INFO Onboard modem ftp disconnected
2021-03-08 15:11:16 [FTP] INFO Onboard modem network disconnected
Via the config it is possible to set the onboard modem to GPS (2G) of LTE (4G) only or GPS (2G) + LTE (4G).
# ----Modem-Settings---- #
Modem:
- id: onboard_modem
port: modem
sim_username:
sim_password:
apn: internet.be
network_technology: GSM
Valid settings are: ‘GSM’, ‘LTE’ and ‘GSM + LTE’. Be aware that not all SIM cards support all network technologies.
Minor patches / fixes:
The EXO table is updated with extra parameters.
The omc2900 output now transmits a ‘telegram’ start and end character. Also, an extra CR and LF are added as done by the OMC045
The modem routine encountered problems when roaming, routine had been improved.
While roaming some timeouts would trigger to early, adjusted to give more time to response.
OMC-048_FW_0.01B1268
Added OMC-2900 serial output
By defining an output driver in the config file, it is possible to convert sensor data to an OMC-2900 output over a serial port. The description on how-to-use is provided in the OMC2900.py file located in the script/modules folder.
Minor patches / fixes:
The EXO and ysi_6_series parameter tables are corrected on tag names/units.
The status indicator of the log format records is reworked to match the omc045III style.
In case a sensor is timed out a record is logged containing an invalid data status indication.
OMC-048_FW_0.01B1228
Added NTP time synchro
The Time can now be synchronized via the NTP protocol. It can be enabled via ‘utc_time_sync’. And optionally the server and port can be provided by the user (defaults: europe.pool.ntp.org, port: 123).
# ----FTP-settings---- #
Ftp:
- id: data_ftp
utc_time_sync: True
utc_time_server: nl.pool.ntp.org
utc_time_server_port: 123
The NTP server is checked after each scheduled FTP task if enabled. After the transmission of the FTP files the NTP server is requested. After a valid NTP package was received the time and date is corrected (taking the loggers UTC offset into account!). To prevent numerous requests to the NTP server a 4 hour timeout was introduces. This prevents unwanted and unnecessary requests to the server.
Added NMEA PTID driver:
The nmea PTID driver is a static driver designed for the Orinoco solo tide sensor. The driver is NMEA based and requires the static input format as:
$PTID,005,10072006,092114,0.08,M,00,12.36,V,1023,hPa,x*19
The parameters which are obtained are: Depth, Status, Battery voltage and Barometric pressure. This driver is statically created due to the absence of a ‘generic nmea’ driver at this moment.
Note: It is important to make sure the logger is awake when the sample is transmitted!
Therefore the sample interval should be set equal to the transmission interval of the sensor and the timeout longer.
Example (transmission ineterval of Orinogo sensor set to 2 minutes).
nmea_ptid:
- id: ptid0
port: serial1
mode: RS232
baudrate: 9600
sample_interval: "0 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58 * * *"
response_timeout: 125
Changes navilock_md6.py:
A minor change in the collection of the nmea data transmitted by the navilock sensor. In case of slower data output the nmea sentences were parsed with prior to last recent data.
Software patches / fixes:
Updated to micropython 1.13
read_line.py refactored using async stream, due to hard fault uart serial problems. The update() now returns the list of sentences, the get() is removed
All Sensor drivers refactored to comply to new read_line.
PAR parameters added to the EXO table, these were somehow forgotten.
ATparser/modem refactored to work with streamreader
Errors on corrupt transmitted directory or unreadable files now delete the ‘problematic files’ instead of trying to re-send.
In driver_manager. In case a sensor times-out the power (if assigned) is force toggled with a off-time given in the sensor driver (default 5 sec).
Some minor changes in the syslog levels to prevent spamming of data or unnecessary messages to the user (such as ‘wake up message’, and ‘usb_wake’)
In navilock. The datetime sync (if enabled) is immediately executed after the receival of the RMC message, this prevents the possibility the time could be synced to an ‘older’ RMC message. In case a logger lost its time the datetime resets to 2000-1-1-1, which was also the date the navilock compared to the first cycle. This caused the datetime not to be updated for 4 hours, changed the starting date to the not-reset date of the logger.
Ysi-6-series. Fix on the temperature bug, multiple spaces as divider are now possible and does not cause the sensor to reject the data message.
Added the dual protocol() function to USB which can change between REPL only and MSC+REPL. This is later to be used for the usb/sd problem fix. This also works for connecting and reconnecting usb during sleep/wake.
The scheduler sometime executed tasks to early. The cron interval is now checked prior to executing the task. If the cron interval is not yet due, it shall go to sleep again for the remainder of the time.
OMC-048_FW_0.01B1038
Added navilock_md6 GPS time synchronisation:
The GPS time can be automatically synced on each valid sample that is received. It is enabled/disabled via ‘utc_time_sync:’ which is driver specific (default is disabled). info messages are reported if the time is synced.
Gps:
- id: GPS_A
utc_time_sync: True
Added UTC offset in configuration:
The system tab can now contain ‘utc_time_offset_hours:’ and ‘utc_time_offset_minutes:’ (zero by default). Positive or negative offsets are given via +x (or x) and -x
# ----System---- #
Omc048:
utc_time_offset_hours: +1
utc_time_offset_minutes: -30
EXO driver wiper:
EXO driver wiper implementation changed from ‘wipe_after_number_of_samples:’ to ‘samples_per_wipe:’. ‘0’ means no wipes, ‘1’ means wipe each sample, ‘10’ means wipe first sample, dont wipe the next 9 samples. etc. The wipe is executed before a measurement(sample), this shall delay the data request with the wipetime (around 1 min)
EXO:
- id: EXO2_0
samples_per_wipe: 10
Log filename change:
The log file name is changed to omc045, the substation name is replaced with the static serial number of the logger. This means substation name in the config is no longer used. log file name is: OMC-045_SYSTEM-ID_SERIALNR_YYMMDD_HHMMSS.txt, example: OMC-045_Scrilo_048000138_201027_134000.txt
OMC-048_FW_0.01B990
- Added Usb_wake:
Usb wake is used to periodicaly wake up the USB interface to check if a usb connection is active. If the usb is found the log file is stopped prior and started after usb negotiations, this reduces the chance of corrupt file detection by windows.
Usb_wake:
- id: usb_wakeup
wake_interval: "0,10,20,30,40,50 * * * *"
The detection of USB wake, FTP and Data interval on startup info messaging: ‘2020-10-29 12:22:39 [LOGGER_MANAGER] INFO Data file create interval detected’ ‘2020-10-29 12:22:39 [FTP] INFO FTP configuration detected: ftp.omc-data-online.com’ ‘2020-10-29 12:22:39 [USB_WAKE] INFO Usb wake interval configuration detected’