Uploading Instrument Data to BigTable

readingTimestamp

defined in Field Definitions

remarks

Publish message with all the data to topic (A)

Push subscription (B) to Cloud Run (B) that processes the data according to the calculations defined in FieldDefs in instrument type

cali1

caliN

status = ingested | loadFail | calcFail | quarantined | published

instrumentType

comments

calcN

Push subscription (A) to Cloud Run (A) that uploads data to BigTable (cf: data)

Yellow: raw readings - fields defined in instrument type's FieldDefs (or instID itself)

Orange: processed by functions - fields and functions defined in instrument type's FieldDefs

Purple: metadata about the readings for this rowkey

instrumentID

dataN

Each color has its own column familt (cf)

calc1

Publish message with calculated data to topic (B)

Push subscription (C) to Cloud Run (C) that uploads the data to BigTable (cf: calc and cali)

Push subscription (D) to Cloud Run (D) that checks for alarm breaches (look up alarm setup in instrument ID), then uploads the data to BigTable (cf: alarmBreach, metadata)

alarmsSetup info

instrumentType

breachedField

instrumentID

instrumentType

instrumentID

readingTimestamp

breachedReading

breachedLevel

breached = true | false

Cyan: row key

Dark purple: alarm breached info - alarms as defined in instrument ID's alarm setup

Cloud Run

If it only uploads data to BigTable, it'll be in green

PubSub

Comment

status = ingested | calcFail