improve systemd unit file

Reviewed-on: #18
Reviewed-by: Philipp Schweizer <Philipp.schw@directbox.com>

.gitignore

@@ -0,0 +1 @@
+docker-compose.yml

README.md

@@ -1,19 +1,99 @@
 # LocationHub
 
-TODO
+**Self-hosted backend for LoRaWAN-based location tracking, supporting multiple location providers such as GNSS, Gateway-Triangulation, WiFi-Triangulation, and BLE-Triangulation.**  
+We recommend using [The Things Network](https://www.thethingsnetwork.org/) (TTN) as middleware for relaying LoRaWAN payloads.
+
+---
+
+## Table of Contents
+1. [Setup](#setup)
+    - [Prerequisites](#prerequisites)
+    - [Database](#database)
+    - [Configuration](#configuration)
+2. [TTN Integration](#ttn-integration)
+3. [Prometheus Metrics](#prometheus-metrics)
+4. [Add a Location Tracker](#add-a-location-tracker)
+    - [Onboard SenseCAP T1000-B](#onboard-sensecap-t1000-b)
+    - [Register SenseCAP T1000-B](#register-sensecap-t1000-b)
+5. [Testing](#testing)
+    - [Testing Webhook](#testing-webhook)
+    - [Emulating Wigle API](#emulating-wigle-api)
+
+---
 
 ## Setup
+
 ### Prerequisites
-- Node.js >= 22.11.0
+Ensure the following dependencies are installed:
-- Maria DB >= 11.6.2
+- **Node.js** >= 22.11.0
+- **MariaDB** >= 11.6.2
+- A web server, such as **nginx**
+
+---
 
 ### Database
-**Change name of database and credentials as you like!**
+**Customize the database name and credentials to your preference.**  
-
+Follow these steps to set up the database:
-- Create new database: `CREATE DATABASE locationhub;`
+- Create new database: `CREATE DATABASE dev_locationhub;`
 - Create new user for database: `GRANT ALL PRIVILEGES ON dev_locationhub.* TO 'dbuser'@'localhost' IDENTIFIED BY '1234';`
 - Import tables: `/usr/bin/mariadb -u dbuser -p1234 dev_locationhub < server/sql/tables.sql`
 
+### Configuration
+- Copy the .env.template file and rename it to .env:
+- Use a strong token/secret for the webhook.
+- Add your [Wigle](http://wigle.net) API token to translate MAC addresses to coordinates
+- Use [systemd](server/scripts/locationhub.service) to start the server.
+
+### TTN Integration
+Create new Webhook for your application. Set base url to `https://your.domain.tld` and enable "Uplink message" to api `/api/ttn/webhook`.
+Add a addidtional header: 
+- Type: `authorization` 
+- Value: `Bearer your-very-secure-token-from-the-env-file`
+
+### Prometheus Metrics
+Use `https://your.domain.tld/api/metrics` to retrieve useful insides for monitoring.
+
+## Add a Location Tracker
+We use the [SenseCAP T1000-B](https://www.seeedstudio.com/SenseCAP-Card-Tracker-T1000-B-p-5698.html) from seeedstudio because of the fair price and multiple location providers. However, you can use any LoRaWAN-enabled tracker that is compatible with TTN and supports the required payload fields.
+
+## Troubleshooting
+Run `journalctl -u locationhub.service -f` to see log output.
+
+### Onboard SenseCAP T1000-B
+1. Download and install the App [SenseCraft](https://play.google.com/store/apps/details?id=cc.seeed.sensecapmate) 
+2. Skip the user account at startup with `Skip` in the upper right corner
+3. Select the `Tracker T1000`
+4. Connect to your SenseCAP T1000-B
+5. In the `Settings` tab select `The Things Network` as Platform under `LoRa`
+6. Copy `Device EUI`, `AppEUI` and `AppKey`
+7. Save LoRa settings
+8. Under `Geolocation` select Geolocation Strategy as `GNSS+Wi-Fi`
+9. Set `GNSS Max Scan Time (s)` to 120
+10. Save Geolocation settings
+11. Exit App with return arrow to trigger re-init of SenseCAP T1000-B
+
+### Register SenseCAP T1000-B
+1. Open your Application in TTN and navigate to `End devices`
+2. Click `Register end device` in the upper right corner
+3. Set input method to `Enter end device specifics manually`
+ - Frequency plan: `Europe 863-870 MHz (SF9 for RX2 - recommended)`
+ - LoRaWAN version: `LoRaWAN Specification 1.0.4`
+ - Regional Parameters version: `RP002 Regional Parameters 1.0.3`
+4. Place as `JoinEUI` the `AppEUI`
+5. Place the `DevEUI`
+6. Place the `AppKey`
+7. Set a name for end device
+8. Click `Register end device`
+9. Check if new end device joins the TTN
+10. Place the [Uplink Payload Formatter](TTN/sensecap_payload_formater.js) as `Custom Javascript formatter` for new end device
+
+## Testing
 ### Testing Webhook
-- To test the webhook use the python script `ttn-webhook-dummy.py` to send prerecorded TTN Uplinks.
+- To test the webhook use the python script [ttn-webhook-dummy.py](server/scripts/ttn-webhook-dummy.py) to send prerecorded TTN Uplinks.
 - To test the script you can use `while true; do echo -e "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\n\r\nSuccess"; nc -l -p 8080 -q 1; done`
+
+### Emulating Wigle API
+- To emulate the Wigle API use the python script [wigle-dummy.py](server/scripts/wigle-dummy.py) to translate MAC addresses to coordinates. 
+
+
+Happy tracking! 🎉

TTN/sensecap_payload_formater.js

@@ -0,0 +1,911 @@
+function decodeUplink (input) {
+    const bytes = input['bytes']
+    const fport = parseInt(input['fPort'])
+    const bytesString = bytes2HexString(bytes)
+    const originMessage = bytesString.toLocaleUpperCase()
+    const decoded = {
+        valid: true,
+        err: 0,
+        payload: bytesString,
+        messages: []
+    }
+    if (fport === 199 || fport === 192) {
+        decoded.messages.push({fport: fport, payload: bytesString})
+        return { data: decoded }
+    }
+    let measurement = messageAnalyzed(originMessage)
+    if (measurement.length === 0) {
+        decoded.valid = false
+        return { data: decoded }
+    }
+
+    for (let message of measurement) {
+        if (message.length === 0) {
+            continue
+        }
+        let elements = []
+        for (let element of message) {
+            if (element.errorCode) {
+                decoded.err = element.errorCode
+                decoded.errMessage = element.error
+            } else {
+                elements.push(element)
+            }
+        }
+        if (elements.length > 0) {
+            decoded.messages.push(elements)
+        }
+    }
+    // decoded.messages = measurement
+    return { data: decoded }
+}
+
+function messageAnalyzed (messageValue) {
+    try {
+        let frames = unpack(messageValue)
+        let measurementResultArray = []
+        for (let i = 0; i < frames.length; i++) {
+            let item = frames[i]
+            let dataId = item.dataId
+            let dataValue = item.dataValue
+            let measurementArray = deserialize(dataId, dataValue)
+            measurementResultArray.push(measurementArray)
+        }
+        return measurementResultArray
+    } catch (e) {
+        return e.toString()
+    }
+}
+
+function unpack (messageValue) {
+    let frameArray = []
+
+    for (let i = 0; i < messageValue.length; i++) {
+        let remainMessage = messageValue
+        let dataId = remainMessage.substring(0, 2).toUpperCase()
+        let dataValue
+        let dataObj = {}
+        let packageLen
+        switch (dataId) {
+            case '01':
+                dataValue = remainMessage.substring(2, 94)
+                messageValue = remainMessage.substring(94)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '02':
+                dataValue = remainMessage.substring(2, 32)
+                messageValue = remainMessage.substring(32)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '03':
+                dataValue = remainMessage.substring(2, 64)
+                messageValue = remainMessage.substring(64)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '04':
+                dataValue = remainMessage.substring(2, 20)
+                messageValue = remainMessage.substring(20)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '05':
+                dataValue = remainMessage.substring(2, 10)
+                messageValue = remainMessage.substring(10)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '06':
+                dataValue = remainMessage.substring(2, 44)
+                messageValue = remainMessage.substring(44)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '07':
+                dataValue = remainMessage.substring(2, 84)
+                messageValue = remainMessage.substring(84)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '08':
+                dataValue = remainMessage.substring(2, 70)
+                messageValue = remainMessage.substring(70)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '09':
+                dataValue = remainMessage.substring(2, 36)
+                messageValue = remainMessage.substring(36)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '0A':
+                dataValue = remainMessage.substring(2, 76)
+                messageValue = remainMessage.substring(76)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '0B':
+                dataValue = remainMessage.substring(2, 62)
+                messageValue = remainMessage.substring(62)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '0C':
+                break
+            case '0D':
+                dataValue = remainMessage.substring(2, 10)
+                messageValue = remainMessage.substring(10)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '0E':
+                packageLen = getInt(remainMessage.substring(8, 10)) * 2 + 10
+                dataValue = remainMessage.substring(2, 8) + remainMessage.substring(10, packageLen)
+                messageValue = remainMessage.substring(packageLen)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '0F':
+                dataValue = remainMessage.substring(2, 34)
+                messageValue = remainMessage.substring(34)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '10':
+                dataValue = remainMessage.substring(2, 26)
+                messageValue = remainMessage.substring(26)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '11':
+                dataValue = remainMessage.substring(2, 28)
+                messageValue = remainMessage.substring(28)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '1A':
+                dataValue = remainMessage.substring(2, 56)
+                messageValue = remainMessage.substring(56)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '1B':
+                dataValue = remainMessage.substring(2, 96)
+                messageValue = remainMessage.substring(96)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '1C':
+                dataValue = remainMessage.substring(2, 82)
+                messageValue = remainMessage.substring(82)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            case '1D':
+                dataValue = remainMessage.substring(2, 40)
+                messageValue = remainMessage.substring(40)
+                dataObj = {
+                    'dataId': dataId, 'dataValue': dataValue
+                }
+                break
+            default:
+                return frameArray
+        }
+        if (dataValue.length < 2) {
+            break
+        }
+        frameArray.push(dataObj)
+    }
+    return frameArray
+}
+
+function deserialize (dataId, dataValue) {
+    let measurementArray = []
+    let eventList = []
+    let measurement = {}
+    let collectTime = 0
+    let groupId = 0
+    let shardFlag = {}
+    let payload = ''
+    let motionId = ''
+    switch (dataId) {
+        case '01':
+            measurementArray = getUpShortInfo(dataValue)
+            measurementArray.push(...getMotionSetting(dataValue.substring(30, 40)))
+            measurementArray.push(...getStaticSetting(dataValue.substring(40, 46)))
+            measurementArray.push(...getShockSetting(dataValue.substring(46, 52)))
+            measurementArray.push(...getTempSetting(dataValue.substring(52, 72)))
+            measurementArray.push(...getLightSetting(dataValue.substring(72, 92)))
+            break
+        case '02':
+            measurementArray = getUpShortInfo(dataValue)
+            break
+        case '03':
+            measurementArray.push(...getMotionSetting(dataValue.substring(0, 10)))
+            measurementArray.push(...getStaticSetting(dataValue.substring(10, 16)))
+            measurementArray.push(...getShockSetting(dataValue.substring(16, 22)))
+            measurementArray.push(...getTempSetting(dataValue.substring(22, 42)))
+            measurementArray.push(...getLightSetting(dataValue.substring(42, 62)))
+            break
+        case '04':
+            let interval = 0
+            let workMode = getInt(dataValue.substring(0, 2))
+            let heartbeatInterval = getMinsByMin(dataValue.substring(4, 8))
+            let periodicInterval = getMinsByMin(dataValue.substring(8, 12))
+            let eventInterval = getMinsByMin(dataValue.substring(12, 16))
+            switch (workMode) {
+                case 0:
+                    interval = heartbeatInterval
+                    break
+                case 1:
+                    interval = periodicInterval
+                    break
+                case 2:
+                    interval = eventInterval
+                    break
+            }
+            measurementArray = [
+                {measurementId: '3940', type: 'Work Mode', measurementValue: workMode},
+                {measurementId: '3942', type: 'Heartbeat Interval', measurementValue: heartbeatInterval},
+                {measurementId: '3943', type: 'Periodic Interval', measurementValue: periodicInterval},
+                {measurementId: '3944', type: 'Event Interval', measurementValue: eventInterval},
+                {measurementId: '3941', type: 'SOS Mode', measurementValue: getSOSMode(dataValue.substring(16, 18))},
+                {measurementId: '3900', type: 'Uplink Interval', measurementValue: interval}
+            ]
+            break;
+        case '05':
+            measurementArray = [
+                {measurementId: '3000', type: 'Battery', measurementValue: getBattery(dataValue.substring(0, 2))},
+                {measurementId: '3940', type: 'Work Mode', measurementValue: getWorkingMode(dataValue.substring(2, 4))},
+                {measurementId: '3965', type: 'Positioning Strategy', measurementValue: getPositioningStrategy(dataValue.substring(4, 6))},
+                {measurementId: '3941', type: 'SOS Mode', measurementValue: getSOSMode(dataValue.substring(6, 8))}
+            ]
+            break
+        case '06':
+            collectTime = getUTCTimestamp(dataValue.substring(8, 16))
+            motionId = getMotionId(dataValue.substring(6, 8))
+            measurementArray = [
+                {measurementId: '4200', timestamp: collectTime, motionId: motionId, type: 'Event Status', measurementValue: getEventStatus(dataValue.substring(0, 6))},
+                {measurementId: '4197', timestamp: collectTime, motionId: motionId, type: 'Longitude', measurementValue: parseFloat(getSensorValue(dataValue.substring(16, 24), 1000000))},
+                {measurementId: '4198', timestamp: collectTime, motionId: motionId, type: 'Latitude', measurementValue: parseFloat(getSensorValue(dataValue.substring(24, 32), 1000000))},
+                {measurementId: '4097', timestamp: collectTime, motionId: motionId, type: 'Air Temperature', measurementValue: getSensorValue(dataValue.substring(32, 36), 10)},
+                {measurementId: '4199', timestamp: collectTime, motionId: motionId, type: 'Light', measurementValue: getSensorValue(dataValue.substring(36, 40))},
+                {measurementId: '3000', timestamp: collectTime, motionId: motionId, type: 'Battery', measurementValue: getBattery(dataValue.substring(40, 42))}
+            ]
+            break
+        case '07':
+            eventList = getEventStatus(dataValue.substring(0, 6))
+            collectTime = getUTCTimestamp(dataValue.substring(8, 16))
+            motionId = getMotionId(dataValue.substring(6, 8))
+            measurementArray = [
+                {measurementId: '4200', timestamp: collectTime, motionId: motionId, type: 'Event Status', measurementValue: getEventStatus(dataValue.substring(0, 6))},
+                {measurementId: '5001', timestamp: collectTime, motionId: motionId, type: 'Wi-Fi Scan', measurementValue: getMacAndRssiObj(dataValue.substring(16, 72))},
+                {measurementId: '4097', timestamp: collectTime, motionId: motionId, type: 'Air Temperature', measurementValue: getSensorValue(dataValue.substring(72, 76), 10)},
+                {measurementId: '4199', timestamp: collectTime, motionId: motionId, type: 'Light', measurementValue: getSensorValue(dataValue.substring(76, 80))},
+                {measurementId: '3000', timestamp: collectTime, motionId: motionId, type: 'Battery', measurementValue: getBattery(dataValue.substring(80, 82))}
+            ]
+            break
+        case '08':
+            collectTime = getUTCTimestamp(dataValue.substring(8, 16))
+            motionId = getMotionId(dataValue.substring(6, 8))
+            measurementArray = [
+                {measurementId: '4200', timestamp: collectTime, motionId: motionId, type: 'Event Status', measurementValue: getEventStatus(dataValue.substring(0, 6))},
+                {measurementId: '5002', timestamp: collectTime, motionId: motionId, type: 'BLE Scan', measurementValue: getMacAndRssiObj(dataValue.substring(16, 58))},
+                {measurementId: '4097', timestamp: collectTime, motionId: motionId, type: 'Air Temperature', measurementValue: getSensorValue(dataValue.substring(58, 62), 10)},
+                {measurementId: '4199', timestamp: collectTime, motionId: motionId, type: 'Light', measurementValue: getSensorValue(dataValue.substring(62, 66))},
+                {measurementId: '3000', timestamp: collectTime, motionId: motionId, type: 'Battery', measurementValue: getBattery(dataValue.substring(66, 68))}
+            ]
+            break
+        case '09':
+            collectTime = getUTCTimestamp(dataValue.substring(8, 16))
+            motionId = getMotionId(dataValue.substring(6, 8))
+            measurementArray = [
+                {measurementId: '4200', timestamp: collectTime, motionId: motionId, type: 'Event Status', measurementValue: getEventStatus(dataValue.substring(0, 6))},
+                {measurementId: '4197', timestamp: collectTime, motionId: motionId, type: 'Longitude', measurementValue: parseFloat(getSensorValue(dataValue.substring(16, 24), 1000000))},
+                {measurementId: '4198', timestamp: collectTime, motionId: motionId, type: 'Latitude', measurementValue: parseFloat(getSensorValue(dataValue.substring(24, 32), 1000000))},
+                {measurementId: '3000', timestamp: collectTime, motionId: motionId, type: 'Battery', measurementValue: getBattery(dataValue.substring(32, 34))}
+            ]
+            break
+        case '0A':
+            collectTime = getUTCTimestamp(dataValue.substring(8, 16))
+            motionId = getMotionId(dataValue.substring(6, 8))
+            measurementArray = [
+                {measurementId: '4200', timestamp: collectTime, motionId, type: 'Event Status', measurementValue: getEventStatus(dataValue.substring(0, 6))},
+                {measurementId: '5001', timestamp: collectTime, motionId, type: 'Wi-Fi Scan', measurementValue: getMacAndRssiObj(dataValue.substring(16, 72))},
+                {measurementId: '3000', timestamp: collectTime, motionId, type: 'Battery', measurementValue: getBattery(dataValue.substring(72, 74))}
+            ]
+            break
+        case '0B':
+            collectTime = getUTCTimestamp(dataValue.substring(8, 16))
+            motionId = getMotionId(dataValue.substring(6, 8))
+            measurementArray = [
+                {measurementId: '4200', timestamp: collectTime, motionId, type: 'Event Status', measurementValue: getEventStatus(dataValue.substring(0, 6))},
+                {measurementId: '5002', timestamp: collectTime, motionId, type: 'BLE Scan', measurementValue: getMacAndRssiObj(dataValue.substring(16, 58))},
+                {measurementId: '3000', timestamp: collectTime, motionId, type: 'Battery', measurementValue: getBattery(dataValue.substring(58, 60))},
+            ]
+            break
+        case '0D':
+            let errorCode = getInt(dataValue)
+            let error = ''
+            switch (errorCode) {
+                case 1:
+                    error = 'FAILED TO OBTAIN THE UTC TIMESTAMP'
+                    break
+                case 2:
+                    error = 'ALMANAC TOO OLD'
+                    break
+                case 3:
+                    error = 'DOPPLER ERROR'
+                    break
+            }
+            measurementArray.push({errorCode, error})
+            break
+        case '0E':
+            shardFlag = getShardFlag(dataValue.substring(0, 2))
+            groupId = getInt(dataValue.substring(2, 6))
+            payload = dataValue.substring(6)
+            measurement = {
+                measurementId: '6152',
+                groupId: groupId,
+                index: shardFlag.index,
+                count: shardFlag.count,
+                type: 'gnss-ng payload',
+                measurementValue: payload
+            }
+            measurementArray.push(measurement)
+            break
+        case '0F':
+            collectTime = getUTCTimestamp(dataValue.substring(8, 16))
+            shardFlag = getShardFlag(dataValue.substring(26, 28))
+            groupId = getInt(dataValue.substring(28, 32))
+            motionId = getMotionId(dataValue.substring(6, 8))
+            measurementArray.push({
+                measurementId: '4200',
+                timestamp: collectTime,
+                motionId,
+                groupId: groupId,
+                index: shardFlag.index,
+                count: shardFlag.count,
+                type: 'Event Status',
+                measurementValue: getEventStatus(dataValue.substring(0, 6))
+            })
+            measurementArray.push({
+                measurementId: '4097',
+                timestamp: collectTime,
+                motionId,
+                groupId: groupId,
+                index: shardFlag.index,
+                count: shardFlag.count,
+                type: 'Air Temperature',
+                measurementValue: getSensorValue(dataValue.substring(16, 20), 10)
+            })
+            measurementArray.push({
+                measurementId: '4199',
+                timestamp: collectTime,
+                motionId,
+                groupId: groupId,
+                index: shardFlag.index,
+                count: shardFlag.count,
+                type: 'Light',
+                measurementValue: getSensorValue(dataValue.substring(20, 24))
+            })
+            measurementArray.push({
+                measurementId: '3000',
+                timestamp: collectTime,
+                motionId,
+                groupId: groupId,
+                index: shardFlag.index,
+                count: shardFlag.count,
+                type: 'Battery',
+                measurementValue: getBattery(dataValue.substring(24, 26))
+            })
+            break
+        case '10':
+            collectTime = getUTCTimestamp(dataValue.substring(8, 16))
+            shardFlag = getShardFlag(dataValue.substring(18, 20))
+            groupId = getInt(dataValue.substring(20, 24))
+            motionId = getMotionId(dataValue.substring(6, 8))
+            measurementArray.push({
+                measurementId: '4200',
+                timestamp: collectTime,
+                motionId,
+                groupId: groupId,
+                index: shardFlag.index,
+                count: shardFlag.count,
+                type: 'Event Status',
+                measurementValue: getEventStatus(dataValue.substring(0, 6))
+            })
+            measurementArray.push({
+                measurementId: '3000',
+                timestamp: collectTime,
+                motionId,
+                groupId: groupId,
+                index: shardFlag.index,
+                count: shardFlag.count,
+                type: 'Battery',
+                measurementValue: getBattery(dataValue.substring(16, 18))
+            })
+            break
+        case '11':
+            collectTime = getUTCTimestamp(dataValue.substring(8, 16))
+            measurementArray.push({
+                measurementId: '3576',
+                timestamp: collectTime,
+                type: 'Positioning Status',
+                measurementValue: getPositingStatus(dataValue.substring(0, 2))
+            })
+            measurementArray.push({
+                timestamp: collectTime,
+                measurementId: '4200',
+                type: 'Event Status',
+                measurementValue: getEventStatus(dataValue.substring(2, 8))
+            })
+            if (!isNaN(parseFloat(getSensorValue(dataValue.substring(16, 20), 10)))) {
+                measurementArray.push({
+                    timestamp: collectTime,
+                    measurementId: '4097',
+                    type: 'Air Temperature',
+                    measurementValue: getSensorValue(dataValue.substring(16, 20), 10)
+                })
+            }
+            if (!isNaN(parseFloat(getSensorValue(dataValue.substring(20, 24))))) {
+                measurementArray.push({
+                    timestamp: collectTime,
+                    measurementId: '4199',
+                    type: 'Light',
+                    measurementValue: getSensorValue(dataValue.substring(20, 24))
+                })
+            }
+            measurementArray.push({
+                timestamp: collectTime,
+                measurementId: '3000',
+                type: 'Battery',
+                measurementValue: getBattery(dataValue.substring(24, 26))
+            })
+            break
+        case '1A':
+            collectTime = getUTCTimestamp(dataValue.substring(8, 16))
+            motionId = getMotionId(dataValue.substring(6, 8))
+            measurementArray = [
+                {measurementId: '4200', timestamp: collectTime, motionId, type: 'Event Status', measurementValue: getEventStatus(dataValue.substring(0, 6))},
+                {measurementId: '4197', timestamp: collectTime, motionId, type: 'Longitude', measurementValue: parseFloat(getSensorValue(dataValue.substring(16, 24), 1000000))},
+                {measurementId: '4198', timestamp: collectTime, motionId, type: 'Latitude', measurementValue: parseFloat(getSensorValue(dataValue.substring(24, 32), 1000000))},
+                {measurementId: '4097', timestamp: collectTime, motionId, type: 'Air Temperature', measurementValue: getSensorValue(dataValue.substring(32, 36), 10)},
+                {measurementId: '4199', timestamp: collectTime, motionId, type: 'Light', measurementValue: getSensorValue(dataValue.substring(36, 40))},
+                {measurementId: '4210', timestamp: collectTime, motionId, type: 'AccelerometerX', measurementValue: getSensorValue(dataValue.substring(40, 44))},
+                {measurementId: '4211', timestamp: collectTime, motionId, type: 'AccelerometerY', measurementValue: getSensorValue(dataValue.substring(44, 48))},
+                {measurementId: '4212', timestamp: collectTime, motionId, type: 'AccelerometerZ', measurementValue: getSensorValue(dataValue.substring(48, 52))},
+                {measurementId: '3000', timestamp: collectTime, motionId, type: 'Battery', measurementValue: getBattery(dataValue.substring(52, 54))},
+            ]
+            break
+        // WIFI定位数据+sensor+三轴+电量
+        case '1B':
+            collectTime = getUTCTimestamp(dataValue.substring(8, 16))
+            motionId = getMotionId(dataValue.substring(6, 8))
+            measurementArray = [
+                {measurementId: '4200', timestamp: collectTime, motionId, type: 'Event Status', measurementValue: getEventStatus(dataValue.substring(0, 6))},
+                {measurementId: '5001', timestamp: collectTime, motionId, type: 'Wi-Fi Scan', measurementValue: getMacAndRssiObj(dataValue.substring(16, 72))},
+                {measurementId: '4097', timestamp: collectTime, motionId, type: 'Air Temperature', measurementValue: getSensorValue(dataValue.substring(72, 76), 10)},
+                {measurementId: '4199', timestamp: collectTime, motionId, type: 'Light', measurementValue: getSensorValue(dataValue.substring(76, 80))},
+                {measurementId: '4210', timestamp: collectTime, motionId, type: 'AccelerometerX', measurementValue: getSensorValue(dataValue.substring(80, 84))},
+                {measurementId: '4211', timestamp: collectTime, motionId, type: 'AccelerometerY', measurementValue: getSensorValue(dataValue.substring(84, 88))},
+                {measurementId: '4212', timestamp: collectTime, motionId, type: 'AccelerometerZ', measurementValue: getSensorValue(dataValue.substring(88, 92))},
+                {measurementId: '3000', timestamp: collectTime, motionId, type: 'Battery', measurementValue: getBattery(dataValue.substring(92, 94))}
+            ]
+            break
+        // BLE定位数据+sensor+三轴+电量
+        case '1C':
+            collectTime = getUTCTimestamp(dataValue.substring(8, 16))
+            motionId = getMotionId(dataValue.substring(6, 8))
+            measurementArray = [
+                {measurementId: '4200', timestamp: collectTime, motionId, type: 'Event Status', measurementValue: getEventStatus(dataValue.substring(0, 6))},
+                {measurementId: '5002', timestamp: collectTime, motionId, type: 'BLE Scan', measurementValue: getMacAndRssiObj(dataValue.substring(16, 58))},
+                {measurementId: '4097', timestamp: collectTime, motionId, type: 'Air Temperature', measurementValue: getSensorValue(dataValue.substring(58, 62), 10)},
+                {measurementId: '4199', timestamp: collectTime, motionId, type: 'Light', measurementValue: getSensorValue(dataValue.substring(62, 66))},
+                {measurementId: '4210', timestamp: collectTime, motionId, type: 'AccelerometerX', measurementValue: getSensorValue(dataValue.substring(66, 70))},
+                {measurementId: '4211', timestamp: collectTime, motionId, type: 'AccelerometerY', measurementValue: getSensorValue(dataValue.substring(70, 74))},
+                {measurementId: '4212', timestamp: collectTime, motionId, type: 'AccelerometerZ', measurementValue: getSensorValue(dataValue.substring(74, 78))},
+                {measurementId: '3000', timestamp: collectTime, motionId, type: 'Battery', measurementValue: getBattery(dataValue.substring(78, 80))}
+            ]
+            break
+        // 定位状态 + sensor+三轴数据上报
+        case '1D':
+            collectTime = getUTCTimestamp(dataValue.substring(8, 16))
+            measurementArray.push({
+                measurementId: '3576',
+                timestamp: collectTime,
+                type: 'Positioning Status',
+                measurementValue: getPositingStatus(dataValue.substring(0, 2))
+            })
+            measurementArray.push({
+                timestamp: collectTime,
+                measurementId: '4200',
+                type: 'Event Status',
+                measurementValue: getEventStatus(dataValue.substring(2, 8))
+            })
+            if (!isNaN(parseFloat(getSensorValue(dataValue.substring(16, 20), 10)))) {
+                measurementArray.push({
+                    timestamp: collectTime,
+                    measurementId: '4097',
+                    type: 'Air Temperature',
+                    measurementValue: getSensorValue(dataValue.substring(16, 20), 10)
+                })
+            }
+            if (!isNaN(parseFloat(getSensorValue(dataValue.substring(20, 24))))) {
+                measurementArray.push({
+                    timestamp: collectTime,
+                    measurementId: '4199',
+                    type: 'Light',
+                    measurementValue: getSensorValue(dataValue.substring(20, 24))
+                })
+            }
+            measurementArray.push({
+                timestamp: collectTime,
+                measurementId: '4210',
+                type: 'AccelerometerX',
+                measurementValue: getSensorValue(dataValue.substring(24, 28))
+            })
+            measurementArray.push({
+                timestamp: collectTime,
+                measurementId: '4211',
+                type: 'AccelerometerY',
+                measurementValue: getSensorValue(dataValue.substring(28, 32))
+            })
+            measurementArray.push({
+                timestamp: collectTime,
+                measurementId: '4212',
+                type: 'AccelerometerZ',
+                measurementValue: getSensorValue(dataValue.substring(32, 36))
+            })
+            measurementArray.push({
+                timestamp: collectTime,
+                measurementId: '3000',
+                type: 'Battery',
+                measurementValue: getBattery(dataValue.substring(36, 38))
+            })
+            break
+    }
+    return measurementArray
+}
+
+function getMotionId (str) {
+    return getInt(str)
+}
+
+function getPositingStatus (str) {
+    let status = getInt(str)
+    switch (status) {
+        case 0:
+            return {id:status, statusName:"Positioning successful."}
+        case 1:
+            return {id:status, statusName:"The GNSS scan timed out and failed to obtain the location."}
+        case 2:
+            return {id:status, statusName:"The Wi-Fi scan timed out and failed to obtain the location."}
+        case 3:
+            return {id:status, statusName:"The Wi-Fi + GNSS scan timed out and failed to obtain the location."}
+        case 4:
+            return {id:status, statusName:"The GNSS + Wi-Fi scan timed out and failed to obtain the location."}
+        case 5:
+            return {id:status, statusName:"The Bluetooth scan timed out and failed to obtain the location."}
+        case 6:
+            return {id:status, statusName:"The Bluetooth + Wi-Fi scan timed out and failed to obtain the location."}
+        case 7:
+            return {id:status, statusName:"The Bluetooth + GNSS scan timed out and failed to obtain the location."}
+        case 8:
+            return {id:status, statusName:"The Bluetooth + Wi-Fi + GNSS scan timed out and failed to obtain the location."}
+        case 9:
+            return {id:status, statusName:"Location Server failed to parse the GNSS location."}
+        case 10:
+            return {id:status, statusName:"Location Server failed to parse the Wi-Fi location."}
+        case 11:
+            return {id:status, statusName:"Location Server failed to parse the Bluetooth location."}
+        case 12:
+            return {id:status, statusName:"Failed to parse the GNSS location due to the poor accuracy."}
+        case 13:
+            return {id:status, statusName:"Time synchronization failed."}
+        case 14:
+            return {id:status, statusName:"Failed to obtain location due to the old Almanac."}
+    }
+    return getInt(str)
+}
+
+function getUpShortInfo (messageValue) {
+    return [
+        {
+            measurementId: '3000', type: 'Battery', measurementValue: getBattery(messageValue.substring(0, 2))
+        }, {
+            measurementId: '3502', type: 'Firmware Version', measurementValue: getSoftVersion(messageValue.substring(2, 6))
+        }, {
+            measurementId: '3001', type: 'Hardware Version', measurementValue: getHardVersion(messageValue.substring(6, 10))
+        }, {
+            measurementId: '3940', type: 'Work Mode', measurementValue: getWorkingMode(messageValue.substring(10, 12))
+        }, {
+            measurementId: '3965', type: 'Positioning Strategy', measurementValue: getPositioningStrategy(messageValue.substring(12, 14))
+        }, {
+            measurementId: '3942', type: 'Heartbeat Interval', measurementValue: getMinsByMin(messageValue.substring(14, 18))
+        }, {
+            measurementId: '3943', type: 'Periodic Interval', measurementValue: getMinsByMin(messageValue.substring(18, 22))
+        }, {
+            measurementId: '3944', type: 'Event Interval', measurementValue: getMinsByMin(messageValue.substring(22, 26))
+        }, {
+            measurementId: '3945', type: 'Sensor Enable', measurementValue: getInt(messageValue.substring(26, 28))
+        }, {
+            measurementId: '3941', type: 'SOS Mode', measurementValue: getSOSMode(messageValue.substring(28, 30))
+        }
+    ]
+}
+
+function getMotionSetting (str) {
+    return [
+        {measurementId: '3946', type: 'Motion Enable', measurementValue: getInt(str.substring(0, 2))},
+        {measurementId: '3947', type: 'Any Motion Threshold', measurementValue: getSensorValue(str.substring(2, 6), 1)},
+        {measurementId: '3948', type: 'Motion Start Interval', measurementValue: getMinsByMin(str.substring(6, 10))},
+    ]
+}
+
+function getStaticSetting (str) {
+    return [
+        {measurementId: '3949', type: 'Static Enable', measurementValue: getInt(str.substring(0, 2))},
+        {measurementId: '3950', type: 'Device Static Timeout', measurementValue: getMinsByMin(str.substring(2, 6))}
+    ]
+}
+
+function getShockSetting (str) {
+    return [
+        {measurementId: '3951', type: 'Shock Enable', measurementValue: getInt(str.substring(0, 2))},
+        {measurementId: '3952', type: 'Shock Threshold', measurementValue: getInt(str.substring(2, 6))}
+    ]
+}
+
+function getTempSetting (str) {
+    return [
+        {measurementId: '3953', type: 'Temp Enable', measurementValue: getInt(str.substring(0, 2))},
+        {measurementId: '3954', type: 'Event Temp Interval', measurementValue: getMinsByMin(str.substring(2, 6))},
+        {measurementId: '3955', type: 'Event Temp Sample Interval', measurementValue: getSecondsByInt(str.substring(6, 10))},
+        {measurementId: '3956', type: 'Temp ThMax', measurementValue: getSensorValue(str.substring(10, 14), 10)},
+        {measurementId: '3957', type: 'Temp ThMin', measurementValue: getSensorValue(str.substring(14, 18), 10)},
+        {measurementId: '3958', type: 'Temp Warning Type', measurementValue: getInt(str.substring(18, 20))}
+    ]
+}
+
+function getLightSetting (str) {
+    return [
+        {measurementId: '3959', type: 'Light Enable', measurementValue: getInt(str.substring(0, 2))},
+        {measurementId: '3960', type: 'Event Light Interval', measurementValue: getMinsByMin(str.substring(2, 6))},
+        {measurementId: '3961', type: 'Event Light Sample Interval', measurementValue: getSecondsByInt(str.substring(6, 10))},
+        {measurementId: '3962', type: 'Light ThMax', measurementValue: getSensorValue(str.substring(10, 14), 10)},
+        {measurementId: '3963', type: 'Light ThMin', measurementValue: getSensorValue(str.substring(14, 18), 10)},
+        {measurementId: '3964', type: 'Light Warning Type', measurementValue: getInt(str.substring(18, 20))}
+    ]
+}
+
+function getShardFlag (str) {
+    let bitStr = getByteArray(str)
+    return {
+        count: parseInt(bitStr.substring(0, 4), 2),
+        index: parseInt(bitStr.substring(4), 2)
+    }
+}
+
+function getBattery (batteryStr) {
+    return loraWANV2DataFormat(batteryStr)
+}
+function getSoftVersion (softVersion) {
+    return `${loraWANV2DataFormat(softVersion.substring(0, 2))}.${loraWANV2DataFormat(softVersion.substring(2, 4))}`
+}
+function getHardVersion (hardVersion) {
+    return `${loraWANV2DataFormat(hardVersion.substring(0, 2))}.${loraWANV2DataFormat(hardVersion.substring(2, 4))}`
+}
+
+function getSecondsByInt (str) {
+    return getInt(str)
+}
+
+function getMinsByMin (str) {
+    return getInt(str)
+}
+
+function getSensorValue (str, dig) {
+    if (str === '8000') {
+        return null
+    } else {
+        return loraWANV2DataFormat(str, dig)
+    }
+}
+
+function bytes2HexString (arrBytes) {
+    var str = ''
+    for (var i = 0; i < arrBytes.length; i++) {
+        var tmp
+        var num = arrBytes[i]
+        if (num < 0) {
+            tmp = (255 + num + 1).toString(16)
+        } else {
+            tmp = num.toString(16)
+        }
+        if (tmp.length === 1) {
+            tmp = '0' + tmp
+        }
+        str += tmp
+    }
+    return str
+}
+function loraWANV2DataFormat (str, divisor = 1) {
+    let strReverse = bigEndianTransform(str)
+    let str2 = toBinary(strReverse)
+    if (str2.substring(0, 1) === '1') {
+        let arr = str2.split('')
+        let reverseArr = arr.map((item) => {
+            if (parseInt(item) === 1) {
+                return 0
+            } else {
+                return 1
+            }
+        })
+        str2 = parseInt(reverseArr.join(''), 2) + 1
+        return parseFloat('-' + str2 / divisor)
+    }
+    return parseInt(str2, 2) / divisor
+}
+
+function bigEndianTransform (data) {
+    let dataArray = []
+    for (let i = 0; i < data.length; i += 2) {
+        dataArray.push(data.substring(i, i + 2))
+    }
+    return dataArray
+}
+
+function toBinary (arr) {
+    let binaryData = arr.map((item) => {
+        let data = parseInt(item, 16)
+            .toString(2)
+        let dataLength = data.length
+        if (data.length !== 8) {
+            for (let i = 0; i < 8 - dataLength; i++) {
+                data = `0` + data
+            }
+        }
+        return data
+    })
+    return binaryData.toString().replace(/,/g, '')
+}
+
+function getSOSMode (str) {
+    return loraWANV2DataFormat(str)
+}
+
+function getMacAndRssiObj (pair) {
+    let pairs = []
+    if (pair.length % 14 === 0) {
+        for (let i = 0; i < pair.length; i += 14) {
+            let mac = getMacAddress(pair.substring(i, i + 12))
+            if (mac) {
+                let rssi = getInt8RSSI(pair.substring(i + 12, i + 14))
+                pairs.push({mac: mac, rssi: rssi})
+            } else {
+                continue
+            }
+        }
+    }
+    return pairs
+}
+
+function getMacAddress (str) {
+    if (str.toLowerCase() === 'ffffffffffff') {
+        return null
+    }
+    let macArr = []
+    for (let i = 1; i < str.length; i++) {
+        if (i % 2 === 1) {
+            macArr.push(str.substring(i - 1, i + 1))
+        }
+    }
+    let mac = ''
+    for (let i = 0; i < macArr.length; i++) {
+        mac = mac + macArr[i]
+        if (i < macArr.length - 1) {
+            mac = mac + ':'
+        }
+    }
+    return mac
+}
+
+function getInt8RSSI (str) {
+    return loraWANV2DataFormat(str)
+}
+
+function getInt (str) {
+    return parseInt(str, 16)
+}
+
+function getEventStatus (str) {
+    // return getInt(str)
+    let bitStr = getByteArray(str)
+    let bitArr = []
+    for (let i = 0; i < bitStr.length; i++) {
+        bitArr[i] = bitStr.substring(i, i + 1)
+    }
+    bitArr = bitArr.reverse()
+    let event = []
+    for (let i = 0; i < bitArr.length; i++) {
+        if (bitArr[i] !== '1') {
+            continue
+        }
+        switch (i){
+            case 0:
+                event.push({id:1, eventName:"Start moving event."})
+                break
+            case 1:
+                event.push({id:2, eventName:"End movement event."})
+                break
+            case 2:
+                event.push({id:3, eventName:"Motionless event."})
+                break
+            case 3:
+                event.push({id:4, eventName:"Shock event."})
+                break
+            case 4:
+                event.push({id:5, eventName:"Temperature event."})
+                break
+            case 5:
+                event.push({id:6, eventName:"Light event."})
+                break
+            case 6:
+                event.push({id:7, eventName:"SOS event."})
+                break
+            case 7:
+                event.push({id:8, eventName:"Press once event."})
+                break
+        }
+    }
+    return event
+}
+
+function getByteArray (str) {
+    let bytes = []
+    for (let i = 0; i < str.length; i += 2) {
+        bytes.push(str.substring(i, i + 2))
+    }
+    return toBinary(bytes)
+}
+
+function getWorkingMode (workingMode) {
+    return getInt(workingMode)
+}
+
+function getPositioningStrategy (strategy) {
+    return getInt(strategy)
+}
+
+function getUTCTimestamp(str){
+    return parseInt(loraWANV2PositiveDataFormat(str)) * 1000
+}
+
+function loraWANV2PositiveDataFormat (str, divisor = 1) {
+    let strReverse = bigEndianTransform(str)
+    let str2 = toBinary(strReverse)
+    return parseInt(str2, 2) / divisor
+}

server/.env.template

@@ -1,6 +1,18 @@
+# Database 
 DB_NAME=""
 DB_USER=""
 DB_PASSWORD=""
-DB_HOST=""
+DB_HOST="localhost"
-DB_DIALECT=""
+DB_DIALECT="mariadb"
-DB_PORT=""
+DB_PORT="3306"
+
+# Server
+PORT="3000" #Port the server runs on, match with your reverse proxy config
+
+# TTN Webhook
+WEBHOOK_TOKEN="" #Token that is placed a the TTN Webhook auth
+
+# Wigle API
+WIGLE_TOKEN="" # Go to account and generate token "Encoded for use"
+WIGLE_BASE_URL="https://api.wigle.net"
+WIGLE_NETWORK_SEARCH="/api/v2/network/search"

server/.gitignore

@@ -309,6 +309,3 @@ cython_debug/
 *.vsix
 
 config.py
-
-#docker
-docker-compose.yml

server/package-lock.json

@@ -7,13 +7,14 @@
     "": {
       "name": "locationhub",
       "version": "1.0.0",
-      "license": "ISC",
+      "license": "MIT",
       "dependencies": {
         "cors": "^2.8.5",
         "dotenv": "^16.4.7",
         "express": "^4.21.2",
         "http-status-codes": "^2.3.0",
         "mariadb": "^3.4.0",
+        "prom-client": "^15.1.3",
         "reflect-metadata": "^0.2.2",
         "sequelize": "^6.37.5",
         "swagger-jsdoc": "^6.2.8",
@@ -120,6 +121,15 @@
       "integrity": "sha512-4JQNk+3mVzK3xh2rqd6RB4J46qUR19azEHBneZyTZM+c456qOrbbM/5xcR8huNCCcbVt7+UmizG6GuUvPvKUYg==",
       "license": "MIT"
     },
+    "node_modules/@opentelemetry/api": {
+      "version": "1.9.0",
+      "resolved": "https://registry.npmjs.org/@opentelemetry/api/-/api-1.9.0.tgz",
+      "integrity": "sha512-3giAOQvZiH5F9bMlMiv8+GSPMeqg0dbaeo58/0SlA9sxSqZhnUtxzX9/2FzyhS9sWQf5S0GJE0AKBrFqjpeYcg==",
+      "license": "Apache-2.0",
+      "engines": {
+        "node": ">=8.0.0"
+      }
+    },
     "node_modules/@scarf/scarf": {
       "version": "1.4.0",
       "resolved": "https://registry.npmjs.org/@scarf/scarf/-/scarf-1.4.0.tgz",
@@ -386,6 +396,12 @@
         "url": "https://github.com/sponsors/sindresorhus"
       }
     },
+    "node_modules/bintrees": {
+      "version": "1.0.2",
+      "resolved": "https://registry.npmjs.org/bintrees/-/bintrees-1.0.2.tgz",
+      "integrity": "sha512-VOMgTMwjAaUG580SXn3LacVgjurrbMme7ZZNYGSSV7mmtY6QQRh0Eg3pwIcntQ77DErK1L0NxkbetjcoXzVwKw==",
+      "license": "MIT"
+    },
     "node_modules/body-parser": {
       "version": "1.20.3",
       "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.20.3.tgz",
@@ -1434,6 +1450,19 @@
         "url": "https://github.com/sponsors/jonschlinkert"
       }
     },
+    "node_modules/prom-client": {
+      "version": "15.1.3",
+      "resolved": "https://registry.npmjs.org/prom-client/-/prom-client-15.1.3.tgz",
+      "integrity": "sha512-6ZiOBfCywsD4k1BN9IX0uZhF+tJkV8q8llP64G5Hajs4JOeVLPCwpPVcpXy3BwYiUGgyJzsJJQeOIv7+hDSq8g==",
+      "license": "Apache-2.0",
+      "dependencies": {
+        "@opentelemetry/api": "^1.4.0",
+        "tdigest": "^0.1.1"
+      },
+      "engines": {
+        "node": "^16 || ^18 || >=20"
+      }
+    },
     "node_modules/proxy-addr": {
       "version": "2.0.7",
       "resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz",
@@ -1873,6 +1902,15 @@
         "express": ">=4.0.0 || >=5.0.0-beta"
       }
     },
+    "node_modules/tdigest": {
+      "version": "0.1.2",
+      "resolved": "https://registry.npmjs.org/tdigest/-/tdigest-0.1.2.tgz",
+      "integrity": "sha512-+G0LLgjjo9BZX2MfdvPfH+MKLCrxlXSYec5DaPYP1fe6Iyhf0/fSmJ0bFiZ1F8BT6cGXl2LpltQptzjXKWEkKA==",
+      "license": "MIT",
+      "dependencies": {
+        "bintrees": "1.0.2"
+      }
+    },
     "node_modules/to-regex-range": {
       "version": "5.0.1",
       "resolved": "https://registry.npmjs.org/to-regex-range/-/to-regex-range-5.0.1.tgz",

server/package.json

@@ -10,7 +10,7 @@
   },
   "keywords": [],
   "author": "Hendrik Schutter, Philipp Schweizer",
-  "license": "ISC",
+  "license": "MIT",
   "devDependencies": {
     "@types/express": "^5.0.0",
     "@types/node": "^22.10.2",
@@ -24,6 +24,7 @@
     "express": "^4.21.2",
     "http-status-codes": "^2.3.0",
     "mariadb": "^3.4.0",
+    "prom-client": "^15.1.3",
     "reflect-metadata": "^0.2.2",
     "sequelize": "^6.37.5",
     "swagger-jsdoc": "^6.2.8",

server/scripts/locationhub.service

@@ -0,0 +1,29 @@
+[Unit]
+Description=LocationHub Service
+Documentation=https://git.mosad.xyz/localhorst/LocationHub
+After=network.target systemd-networkd-wait-online.service mysqld.service
+
+[Service]
+Type=simple
+User=locationhub
+Group=locationhub
+WorkingDirectory=/home/locationhub/git/LocationHub/server/
+
+# Combine commands for build and start
+ExecStart=/bin/bash -c "/usr/bin/npm run build && /usr/bin/npm run start"
+
+# Restart policies
+Restart=on-failure
+RestartSec=5s
+
+# Logging configuration
+StandardOutput=journal
+StandardError=journal
+SyslogIdentifier=locationhub
+
+# Resource control (optional but helps stability)
+MemoryLimit=512M
+CPUQuota=50%
+
+[Install]
+WantedBy=multi-user.target

server/scripts/ttn-webhook-dummy.py

@@ -9,12 +9,16 @@ import json
 import argparse
 import random
 
-def send_post_request(uri, data):
+def send_post_request(uri, data, token):
+    headers = {
+        "Authorization": f"Bearer {token}",
+        "Content-Type": "application/json",
+    }
     try:
-        requests.post(uri, json=data, timeout=1)
+        response = requests.post(uri, json=data, timeout=1, headers=headers)
+        print("Return code: " + str(response.status_code))
     except requests.exceptions.RequestException as e:
-        pass
+        print(e)
-
 
 def main():
     parser = argparse.ArgumentParser(
@@ -25,6 +29,11 @@ def main():
         type=str,
         help="The URI to send POST requests to (e.g., http://127.0.0.1:8080/api)",
     )
+    parser.add_argument(
+        "token",
+        type=str,
+        help="Bearer authorization token)",
+    )
     parser.add_argument(
         "directory",
         type=str,
@@ -41,6 +50,16 @@ def main():
     )
 
     args = parser.parse_args()
+    if args.mode == "send_one":
+        if os.path.isfile(args.directory) and args.directory.endswith(".json"):
+            with open(args.directory, "r", encoding="utf-8") as file:
+                try:
+                    data = json.load(file)
+                    print(f"Sending {args.directory} to {args.uri}")
+                    send_post_request(args.uri, data, args.token)
+                except json.JSONDecodeError as e:
+                    print(f"Error reading {args.directory}: {e}")
+            return
 
     if not os.path.exists(args.directory):
         print(f"Directory {args.directory} does not exist.")
@@ -58,7 +77,7 @@ def main():
                 try:
                     data = json.load(file)
                     print(f"Sending {filename} to {args.uri}")
-                    send_post_request(args.uri, data)
+                    send_post_request(args.uri, data, args.token)
                 except json.JSONDecodeError as e:
                     print(f"Error reading {filename}: {e}")
 
@@ -69,7 +88,7 @@ def main():
                 try:
                     data = json.load(file)
                     print(f"Sending {filename} to {args.uri}")
-                    send_post_request(args.uri, data)
+                    send_post_request(args.uri, data, args.token)
                     input("Press Enter to send the next file...")
                 except json.JSONDecodeError as e:
                     print(f"Error reading {filename}: {e}")
@@ -82,11 +101,10 @@ def main():
                 try:
                     data = json.load(file)
                     print(f"Sending {filename} to {args.uri}")
-                    send_post_request(args.uri, data)
+                    send_post_request(args.uri, data, args.token)
                     input("Press Enter to send another random file...")
                 except json.JSONDecodeError as e:
                     print(f"Error reading {filename}: {e}")
 
-
 if __name__ == "__main__":
     main()

server/scripts/wigle-dummy.py

@@ -0,0 +1,88 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+""" Author:                     Hendrik Schutter, mail@hendrikschutter.com
+"""
+
+import requests
+import os
+import json
+import argparse
+import random
+import http.server
+import json
+from urllib.parse import urlparse, parse_qs
+
+port = 8000
+
+def generateDummyResponse(netid):
+    response_payload = {
+        "success": True,
+        "totalResults": 1,
+        "first": 0,
+        "last": 0,
+        "resultCount": 1,
+        "results": [
+            {
+                "trilat": random.uniform(-90, 90),
+                "trilong": random.uniform(-180, 180),
+                "ssid": "Wifi-Name",
+                "qos": 0,
+                "transid": "string",
+                "firsttime": "2025-01-02T16:48:28.368Z",
+                "lasttime": "2025-01-02T16:48:28.368Z",
+                "lastupdt": "2025-01-02T16:48:28.368Z",
+                "netid": netid,
+                "name": "string",
+                "type": "string",
+                "comment": "string",
+                "wep": "string",
+                "bcninterval": 0,
+                "freenet": "string",
+                "dhcp": "string",
+                "paynet": "string",
+                "userfound": False,
+                "channel": 0,
+                "rcois": "string",
+                "encryption": "none",
+                "country": "string",
+                "region": "string",
+                "road": "string",
+                "city": "string",
+                "housenumber": "string",
+                "postalcode": "string",
+            }
+        ],
+        "searchAfter": "string",
+        "search_after": 0,
+    }
+
+    return response_payload
+
+class SimpleHTTPRequestHandler(http.server.BaseHTTPRequestHandler):
+    def do_GET(self):
+        # Parse the URL and query parameters
+        parsed_url = urlparse(self.path)
+        if parsed_url.path == "/api/v2/network/search":
+            query_params = parse_qs(parsed_url.query)
+            netid = query_params.get("netid", [""])[0]
+
+            # Send response headers
+            self.send_response(200)
+            self.send_header("Content-Type", "application/json")
+            self.end_headers()
+
+            # Send the JSON response
+            self.wfile.write(json.dumps(generateDummyResponse(netid)).encode("utf-8"))
+        else:
+            # Handle 404 Not Found
+            self.send_response(404)
+            self.end_headers()
+            self.wfile.write(b"Not Found")
+
+def main():
+    server = http.server.HTTPServer(("127.0.0.1", port), SimpleHTTPRequestHandler)
+    print(f"Server running on http://127.0.0.1:{port}/api/v2/network/search'...")
+    server.serve_forever()
+
+if __name__ == "__main__":
+    main()

server/sql/tables.sql

@@ -5,10 +5,8 @@ CREATE TABLE IF NOT EXISTS lp_ttn_end_device_uplinks (
     dev_eui VARCHAR(255),
     join_eui VARCHAR(255),
     dev_addr VARCHAR(255),
-    received_at_utc DATE,
+    received_at_utc DATE NOT NULL,
     battery NUMERIC,
-    latitude DOUBLE,
-    longitude DOUBLE,
     created_at_utc TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
     updated_at_utc TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
 );
@@ -18,13 +16,31 @@ CREATE TABLE IF NOT EXISTS wifi_scan (
     lp_ttn_end_device_uplinks_id UUID,
     mac VARCHAR(255),
     rssi NUMERIC,
-    latitude DOUBLE,
+    scanned_at_utc TIMESTAMP NOT NULL,
-    longitude DOUBLE,
     created_at_utc TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
     updated_at_utc TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
     FOREIGN KEY (lp_ttn_end_device_uplinks_id) REFERENCES lp_ttn_end_device_uplinks(lp_ttn_end_device_uplinks_id)
 );
 
+CREATE TABLE IF NOT EXISTS wifi_location (
+    mac VARCHAR(255) PRIMARY KEY,
+    latitude DOUBLE,
+    longitude DOUBLE,
+    request_limit_exceeded boolean NOT NULL,
+    location_not_resolvable boolean NOT NULL,
+    created_at_utc TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
+    updated_at_utc TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
+);
+
+CREATE TABLE IF NOT EXISTS wifi_location_history (
+    wifi_location_history_id UUID PRIMARY KEY,
+    mac VARCHAR(255),
+    latitude DOUBLE NOT NULL,
+    longitude DOUBLE NOT NULL,
+    created_at_utc TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
+    updated_at_utc TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
+);
+
 CREATE TABLE IF NOT EXISTS ttn_gateway_reception (
     ttn_gateway_reception_id UUID PRIMARY KEY,
     lp_ttn_end_device_uplinks_id UUID,
@@ -46,6 +62,9 @@ CREATE TABLE IF NOT EXISTS location (
     wifi_longitude DOUBLE,
     gnss_latitude DOUBLE,
     gnss_longitude DOUBLE,
+    ttn_gw_latitude DOUBLE,
+    ttn_gw_longitude DOUBLE,
+    gnss_location_at_utc TIMESTAMP,
     created_at_utc TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
     updated_at_utc TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
     FOREIGN KEY (lp_ttn_end_device_uplinks_id) REFERENCES lp_ttn_end_device_uplinks(lp_ttn_end_device_uplinks_id)

server/src/controller/metricsController.ts

@@ -0,0 +1,63 @@
+import express, { Request, Response } from "express";
+import { container } from "tsyringe";
+import { Counter, Gauge, collectDefaultMetrics, register } from "prom-client";
+import { LocationService } from "../services/locationService";
+import { WifiLocationService } from "../services/wifiLocationService";
+import { TtnGatewayReceptionService } from "../services/ttnGatewayReceptionService";
+import { MetricsService } from "../services/metricsService";
+
+const router = express.Router();
+const locationService = container.resolve(LocationService);
+const wifiLocationService = container.resolve(WifiLocationService);
+const ttnGatewayReceptionService = container.resolve(TtnGatewayReceptionService);
+const metricsService = container.resolve(MetricsService);
+
+const requestCounter = metricsService.getRequestCounter();
+const locationsTotal = metricsService.getLocationsTotal();
+const gnssLocationsTotal = metricsService.getGnssLocationsTotal()
+const wifiLocationTotal = metricsService.getWifiLocationTotal();
+const wifiLocationNotResolvable = metricsService.getWifiLocationResolvable();
+const wifiLocationRequestLimitExceeded = metricsService.getLWifiLocationRequestLimitExceeded();
+const ttnGatewayReceptions = metricsService.getTnnGatewayReceptions();
+
+// Define the metrics endpoint
+router.get("/", async (req: Request, res: Response) => {
+  try {
+    const [
+      allLocations,
+      gnssLocations,
+      allWifiLocations,
+      wifiLocationsNotResolvable,
+      wifiLocationsRequestLimitExceeded,
+      allTtnGatewayReceptions
+    ] = await Promise.all([
+      locationService.getAllLocations(),
+      locationService.getAllGnssLocations(),
+      wifiLocationService.getAllWifiLocations(),
+      wifiLocationService.getAllWifiLocationsByNotResolvable(),
+      wifiLocationService.getAllWifiLocationsByRequestLimitExceeded(),
+      ttnGatewayReceptionService.getAllGatewayReceptions()
+    ]);
+
+    locationsTotal.set(allLocations.length);
+    gnssLocationsTotal.set(gnssLocations.length);
+    wifiLocationTotal.set(allWifiLocations.length);
+    wifiLocationNotResolvable.set(wifiLocationsNotResolvable.length);
+    wifiLocationRequestLimitExceeded.set(wifiLocationsRequestLimitExceeded.length);
+    ttnGatewayReceptions.set(allTtnGatewayReceptions.length);
+
+    // Increment the counter with labels
+    requestCounter.inc({ method: req.method, route: req.route.path, status: 200 });
+
+    // Expose metrics in Prometheus format
+    res.set("Content-Type", register.contentType);
+    res.send(await register.metrics());
+  } catch (error) {
+    // Increment the counter for errors
+    requestCounter.inc({ method: req.method, route: req.route.path, status: 500 });
+    console.error("Error running metrics endpoint:", error);
+    res.status(500).json({ error: "Error running metrics endpoint" });
+  }
+});
+
+export default router;

server/src/controller/ttnController.ts

@@ -1,12 +1,10 @@
 import express, { Request, Response } from "express";
+import { StatusCodes } from "http-status-codes";
 import { container } from "tsyringe";
-import { domainEventEmitter } from "../config/eventEmitter";
+import { authenticateHeader } from "../middleware/authentificationMiddleware";
-import {
-  TtnMessageReceivedEvent,
-  TtnMessageReceivedEventName,
-} from "../event/ttnMessageReceivedEvent";
 import { validateData } from "../middleware/validationMiddleware";
 import { TtnMessage } from "../models/ttnMessage";
+import { LocationService } from "../services/locationService";
 import { LpTtnEndDeviceUplinksService } from "../services/lpTtnEndDeviceUplinksService";
 import { TtnGatewayReceptionService } from "../services/ttnGatewayReceptionService";
 import { WifiScanService } from "../services/wifiScanService";
@@ -19,15 +17,17 @@ const ttnGatewayReceptionService = container.resolve(
   TtnGatewayReceptionService
 );
 const wifiScanService = container.resolve(WifiScanService);
+const locationService = container.resolve(LocationService);
 
 const router = express.Router();
 
 router.post(
   "/webhook",
-  validateData(ttnMessageValidator),
+  [authenticateHeader, validateData(ttnMessageValidator)],
   async (req: Request, res: Response) => {
     try {
       const message = req.body as TtnMessage;
+
       const { lp_ttn_end_device_uplinks_id } =
         await lpTtnEndDeviceUplinksService.createUplink({
           device_id: message.end_device_ids.device_id,
@@ -40,22 +40,40 @@ router.post(
           battery: message.uplink_message.decoded_payload?.messages[0].find(
             (e) => e.type === "Battery"
           )?.measurementValue,
-          latitude: message.uplink_message.decoded_payload?.messages[0].find(
-            (e) => e.type === "Latitude"
-          )?.measurementValue,
-          longitude: message.uplink_message.decoded_payload?.messages[0].find(
-            (e) => e.type === "Longitude"
-          )?.measurementValue,
         });
 
+      const messageData = message.uplink_message.decoded_payload?.messages[0];
+
+      const latitudeData = messageData?.find((e) => e.type === "Latitude");
+      const longitudeData = messageData?.find((e) => e.type === "Longitude");
+
+      const gnnsLocation = {
+        latitude: latitudeData?.measurementValue,
+        longitude: longitudeData?.measurementValue,
+      };
+
+      const gnssTimestamp = {
+        timestamp: latitudeData?.timestamp
+          ? new Date(latitudeData.timestamp)
+          : longitudeData?.timestamp
+          ? new Date(longitudeData.timestamp)
+          : undefined,
+      };
+
+      const wifiMessage =
+        message.uplink_message.decoded_payload?.messages[0].find(
+          (e) => e.type === "Wi-Fi Scan"
+        );
+
       const wifiScans =
-        message.uplink_message.decoded_payload?.messages[0]
+        wifiMessage?.measurementValue?.map((w) => ({
-          .find((e) => e.type === "Wi-Fi Scan")
+          lp_ttn_end_device_uplinks_id,
-          ?.measurementValue?.map((w) => ({
+          mac: w.mac,
-            lp_ttn_end_device_uplinks_id,
+          rssi: w.rssi,
-            mac: w.mac,
+          scanned_at_utc: wifiMessage?.timestamp
-            rssi: w.rssi,
+            ? new Date(wifiMessage.timestamp)
-          })) ?? [];
+            : undefined,
+        })) ?? [];
 
       const ttnGatewayReceptions = message.uplink_message.rx_metadata.map(
         (g) => ({
@@ -63,36 +81,48 @@ router.post(
           gateway_id: g.gateway_ids.gateway_id,
           eui: g.gateway_ids.eui,
           rssi: g.rssi,
-          latitude: g.location.latitude,
+          latitude: g.location?.latitude,
-          longitude: g.location.longitude,
+          longitude: g.location?.longitude,
-          altitude: g.location.altitude,
+          altitude: g.location?.altitude,
         })
       );
 
-      const event: TtnMessageReceivedEvent = {
+      const createDatabaseEntries = async () => {
-        lp_ttn_end_device_uplinks_id,
+        const [wifiResults, gatewayResults] = await Promise.all([
-        wifis: wifiScans.map((w) => ({ mac: w.mac, rssi: w.rssi })),
+          wifiScanService.createWifiScans(wifiScans),
-        ttnGateways: ttnGatewayReceptions.map((g) => ({
+          ttnGatewayReceptionService.filterAndInsertGatewayReception(
-          rssi: g.rssi,
+            ttnGatewayReceptions
-          altitude: g.altitude,
+          ),
-          latitude: g.latitude,
+        ]);
-          longitude: g.longitude,
+
-        })),
+        locationService.createLocationFromTriangulation({
+          lp_ttn_end_device_uplinks_id,
+          wifi: wifiResults.map(({ mac, rssi }) => ({
+            mac,
+            rssi,
+          })),
+          ttn_gw: gatewayResults.map(({ latitude, longitude, rssi }) => ({
+            latitude,
+            longitude,
+            rssi,
+          })),
+          gnss:
+            gnnsLocation.latitude && gnnsLocation.longitude
+              ? {
+                  latitude: gnnsLocation.latitude,
+                  longitude: gnnsLocation.longitude,
+                }
+              : undefined,
+          gnss_timestamp: gnssTimestamp.timestamp,
+        });
       };
-
+      createDatabaseEntries().then();
-      domainEventEmitter.emit(TtnMessageReceivedEventName, event);
+      res.status(StatusCodes.OK).send();
-
-      await Promise.all([
-        wifiScanService.createWifiScans(wifiScans),
-        ttnGatewayReceptionService.createGatewayReceptions(
-          ttnGatewayReceptions
-        ),
-      ]);
-
-      res.status(200);
     } catch (error) {
       console.log(error);
-      res.status(500).json({ error: "Error creating uplink" });
+      res
+        .status(StatusCodes.INTERNAL_SERVER_ERROR)
+        .json({ error: "Error creating uplink" });
     }
   }
 );

server/src/controller/ttnGatewayReceptionController.ts

@@ -1,6 +1,6 @@
 import express, { Request, Response } from "express";
-import { TtnGatewayReceptionService } from "../services/ttnGatewayReceptionService";
 import { container } from "tsyringe";
+import { TtnGatewayReceptionService } from "../services/ttnGatewayReceptionService";
 
 const ttnGatewayReceptionService = container.resolve(
   TtnGatewayReceptionService
@@ -35,7 +35,7 @@ router.get("/:id", async (req: Request, res: Response) => {
 router.post("/", async (req: Request, res: Response) => {
   try {
     const newGatewayReception =
-      await ttnGatewayReceptionService.createGatewayReception(req.body);
+      await ttnGatewayReceptionService.createTtnGatewayReception(req.body);
     res.status(201).json(newGatewayReception);
   } catch (error) {
     res.status(500).json({ error: "Error creating gateway reception" });
@@ -46,7 +46,10 @@ router.put("/:id", async (req: Request, res: Response) => {
   try {
     const { id } = req.params;
     const updatedGatewayReception =
-      await ttnGatewayReceptionService.updateGatewayReception(id, req.body);
+      await ttnGatewayReceptionService.updateGatewayReception({
+        ...req.body,
+        ttn_gateway_reception_id: id,
+      });
     if (!updatedGatewayReception) {
       res.status(404).json({ error: "Gateway reception not found" });
       return;

server/src/controller/wifiLocationController.ts

@@ -0,0 +1,32 @@
+import express, { Request, Response } from "express";
+import { container } from "tsyringe";
+import { WifiLocationService } from "../services/wifiLocationService";
+
+const wifiLocationService = container.resolve(WifiLocationService);
+const router = express.Router();
+
+router.get("/", async (req: Request, res: Response) => {
+  try {
+    const wifiLocations = await wifiLocationService.getAllWifiLocations();
+    res.status(200).json(wifiLocations);
+  } catch (error) {
+    console.log(error);
+    res.status(500).json({ error: "Error retrieving wifi location" });
+  }
+});
+
+router.delete("/:id", async (req: Request, res: Response) => {
+  try {
+    const { id } = req.params;
+    const deleted = await wifiLocationService.deleteWifiLocation(id);
+    if (!deleted) {
+      res.status(404).json({ error: "Wifi Location not found" });
+      return;
+    }
+    res.status(204).send();
+  } catch (error) {
+    res.status(500).json({ error: "Error deleting wifi location" });
+  }
+});
+
+export default router;

server/src/controller/wifiLocationHistoryController.ts

@@ -0,0 +1,35 @@
+import express, { Request, Response } from "express";
+import { container } from "tsyringe";
+import { WifiLocationHistoryService } from "../services/wifiLocationHistoryService";
+
+const wifiLocationHistoryService = container.resolve(
+  WifiLocationHistoryService
+);
+const router = express.Router();
+
+router.get("/", async (req: Request, res: Response) => {
+  try {
+    const wifiLocationHistory =
+      await wifiLocationHistoryService.getAllWifiLocationHistories();
+    res.status(200).json(wifiLocationHistory);
+  } catch (error) {
+    res.status(500).json({ error: "Error retrieving wifi location history" });
+  }
+});
+
+router.get("/:id", async (req: Request, res: Response) => {
+  try {
+    const { id } = req.params;
+    const wifiLocationHistory =
+      await wifiLocationHistoryService.getWifiLocationHistoryById(id);
+    if (!wifiLocationHistory) {
+      res.status(404).json({ error: "Wifi location history not found" });
+      return;
+    }
+    res.status(200).json(wifiLocationHistory);
+  } catch (error) {
+    res.status(500).json({ error: "Error retrieving wifi location history" });
+  }
+});
+
+export default router;

server/src/controller/wifiScanController.ts

@@ -40,7 +40,10 @@ router.post("/", async (req: Request, res: Response) => {
 router.put("/:id", async (req: Request, res: Response) => {
   try {
     const { id } = req.params;
-    const updatedWifiScan = await wifiScanService.updateWifiScan(id, req.body);
+    const updatedWifiScan = await wifiScanService.updateWifiScan({
+      ...req.body,
+      wifi_scan_id: id,
+    });
     if (!updatedWifiScan) {
       res.status(404).json({ error: "Wifi scan not found" });
       return;

server/src/event/ttnMessageReceivedEvent.ts

@@ -1,14 +0,0 @@
-export const TtnMessageReceivedEventName = "TtnMessageReceived";
-export type TtnMessageReceivedEvent = {
-  lp_ttn_end_device_uplinks_id: string;
-  wifis: {
-    mac: string;
-    rssi: number;
-  }[];
-  ttnGateways: {
-    rssi: number;
-    latitude: number;
-    longitude: number;
-    altitude: number;
-  }[];
-};

server/src/eventHandler/ttnMessageReceivedEventHandler.ts

@@ -1,13 +0,0 @@
-import { domainEventEmitter } from "../config/eventEmitter";
-import {
-  TtnMessageReceivedEvent,
-  TtnMessageReceivedEventName,
-} from "../event/ttnMessageReceivedEvent";
-
-domainEventEmitter.on(
-  TtnMessageReceivedEventName,
-  async (event: TtnMessageReceivedEvent) => {
-    console.log(event);
-    // TODO Hendrik 🚀
-  }
-);

server/src/index.ts

@@ -1,14 +1,16 @@
 import dotenv from "dotenv";
 import express from "express";
 import "reflect-metadata";
-import "./eventHandler/ttnMessageReceivedEventHandler";
 const cors = require("cors");
 
 import locationRoutes from "./controller/locationController";
 import lpTtnEndDeviceUplinksRoutes from "./controller/lpTtnEndDeviceUplinksController";
 import ttnRoutes from "./controller/ttnController";
 import ttnGatewayReceptionRoutes from "./controller/ttnGatewayReceptionController";
+import wifiLocationRoutes from "./controller/wifiLocationController";
+import wifiLocationHistoryRoutes from "./controller/wifiLocationHistoryController";
 import wifiScanRoutes from "./controller/wifiScanController";
+import metricsRoutes from "./controller/metricsController";
 
 dotenv.config();
 
@@ -19,11 +21,14 @@ app.use(cors());
 app.use(express.json());
 
 app.use("/api/lp-ttn-end-device-uplinks", lpTtnEndDeviceUplinksRoutes);
-app.use("/api/wifi-scans", wifiScanRoutes);
 app.use("/api/ttn-gateway-receptions", ttnGatewayReceptionRoutes);
+app.use("/api/wifi-location-history", wifiLocationHistoryRoutes);
+app.use("/api/wifi-location", wifiLocationRoutes);
+app.use("/api/wifi-scans", wifiScanRoutes);
 app.use("/api/locations", locationRoutes);
 app.use("/api/ttn", ttnRoutes);
+app.use("/api/metrics", metricsRoutes);
 
 app.listen(PORT, () => {
-  console.log(`🚀 Server läuft auf http://localhost:${PORT}`);
+  console.log(`🚀 Server runs here: http://localhost:${PORT}`);
 });

server/src/middleware/authentificationMiddleware.ts

@@ -0,0 +1,41 @@
+import { NextFunction, Request, Response } from "express";
+import { StatusCodes } from "http-status-codes";
+
+const validateBearerToken = (authorizationHeader: string | undefined): boolean => {
+    if (!authorizationHeader) {
+        console.log("Authorization header is missing!");
+        return false;
+    }
+
+    const token = authorizationHeader.split(' ')[1]; // Extract token after 'Bearer'
+    if (!token) {
+        console.log("Bearer token is missing!");
+        return false;
+    }
+
+    if (token !== process.env.WEBHOOK_TOKEN) {
+        console.log("Bearer token is incorrect!");
+        return false;
+    }
+
+    return true;
+};
+
+export function authenticateHeader(req: Request, res: Response, next: NextFunction) {
+    try {
+        const authorizationHeader = req.headers['authorization'];
+
+        if (!validateBearerToken(authorizationHeader as string)) {
+            res.status(StatusCodes.UNAUTHORIZED).json({ error: "Authentication failed" });
+            return;
+        }
+        //console.log("Bearer token is correct!");
+        next();
+    } catch (error) {
+        res.status(StatusCodes.INTERNAL_SERVER_ERROR)
+            .json({ error: "Internal Server Error" });
+    }
+};
+
+
+

server/src/models/location.ts

@@ -8,6 +8,9 @@ export class Location extends Model {
   public wifi_longitude!: number;
   public gnss_latitude!: number;
   public gnss_longitude!: number;
+  public ttn_gw_latitude!: number;
+  public ttn_gw_longitude!: number;
+  public gnss_location_at_utc!: Date;
   public created_at_utc!: Date;
   public updated_at_utc!: Date;
 }
@@ -18,27 +21,30 @@ Location.init(
       type: DataTypes.UUID,
       defaultValue: DataTypes.UUIDV4,
       primaryKey: true,
-      allowNull: false,
     },
     lp_ttn_end_device_uplinks_id: {
       type: DataTypes.UUID,
-      allowNull: false,
     },
     wifi_latitude: {
       type: DataTypes.NUMBER,
-      allowNull: true,
     },
     wifi_longitude: {
       type: DataTypes.NUMBER,
-      allowNull: true,
     },
     gnss_latitude: {
       type: DataTypes.NUMBER,
-      allowNull: true,
     },
     gnss_longitude: {
       type: DataTypes.NUMBER,
-      allowNull: true,
+    },
+    ttn_gw_latitude: {
+      type: DataTypes.NUMBER,
+    },
+    ttn_gw_longitude: {
+      type: DataTypes.NUMBER,
+    },
+    gnss_location_at_utc: {
+      type: DataTypes.DATE,
     },
     created_at_utc: {
       type: DataTypes.DATE,

server/src/models/lpTtnEndDeviceUplinks.ts

@@ -10,8 +10,6 @@ export class LpTtnEndDeviceUplinks extends Model {
   public dev_addr!: string;
   public received_at_utc!: Date;
   public battery!: number;
-  public latitude!: number;
-  public longitude!: number;
   public created_at_utc!: Date;
   public updated_at_utc!: Date;
 }
@@ -30,35 +28,21 @@ LpTtnEndDeviceUplinks.init(
     },
     application_ids: {
       type: DataTypes.STRING,
-      allowNull: true,
     },
     dev_eui: {
       type: DataTypes.STRING,
-      allowNull: true,
     },
     join_eui: {
       type: DataTypes.STRING,
-      allowNull: true,
     },
     dev_addr: {
       type: DataTypes.STRING,
-      allowNull: true,
     },
     received_at_utc: {
       type: DataTypes.DATE,
-      allowNull: true,
     },
     battery: {
       type: DataTypes.NUMBER,
-      allowNull: true,
-    },
-    latitude: {
-      type: DataTypes.NUMBER,
-      allowNull: true,
-    },
-    longitude: {
-      type: DataTypes.NUMBER,
-      allowNull: true,
     },
     created_at_utc: {
       type: DataTypes.DATE,

server/src/models/ttnGatewayReception.ts

@@ -32,23 +32,18 @@ TtnGatewayReception.init(
     },
     eui: {
       type: DataTypes.STRING,
-      allowNull: false,
     },
     rssi: {
       type: DataTypes.NUMBER,
-      allowNull: true,
     },
     latitude: {
       type: DataTypes.NUMBER,
-      allowNull: true,
     },
     longitude: {
       type: DataTypes.NUMBER,
-      allowNull: true,
     },
     altitude: {
       type: DataTypes.NUMBER,
-      allowNull: true,
     },
     created_at_utc: {
       type: DataTypes.DATE,

server/src/models/ttnMessage.ts

@@ -6,14 +6,14 @@ export interface TtnMessage {
     };
     dev_eui: string;
     join_eui: string;
-    dev_addr: string;
+    dev_addr?: string;
   };
   correlation_ids: string[];
   received_at: string;
   uplink_message: {
-    session_key_id: string;
+    session_key_id?: string;
     f_port?: number;
-    f_cnt: number;
+    f_cnt?: number;
     frm_payload?: string;
     decoded_payload?: {
       err: number;
@@ -22,8 +22,8 @@ export interface TtnMessage {
           {
             measurementId: "4200";
             measurementValue: any[];
-            motionId: number;
+            motionId?: number;
-            timestamp: number;
+            timestamp?: number;
             type: "Event Status";
           },
           {
@@ -32,29 +32,29 @@ export interface TtnMessage {
               mac: string;
               rssi: number;
             }[];
-            motionId: number;
+            motionId?: number;
-            timestamp: number;
+            timestamp?: number;
             type: "Wi-Fi Scan";
           },
           {
             measurementId: "3000";
             measurementValue: number;
-            motionId: number;
+            motionId?: number;
-            timestamp: number;
+            timestamp?: number;
             type: "Battery";
           },
           {
             measurementId: "4197";
             measurementValue: number;
-            motionId: number;
+            motionId?: number;
-            timestamp: number;
+            timestamp?: number;
             type: "Longitude";
           },
           {
             measurementId: "4198";
             measurementValue: number;
-            motionId: number;
+            motionId?: number;
-            timestamp: number;
+            timestamp?: number;
             type: "Latitude";
           }
         ]
@@ -67,44 +67,44 @@ export interface TtnMessage {
         gateway_id: string;
         eui?: string;
       };
-      time: string;
+      time?: string;
       timestamp?: number;
       rssi: number;
       channel_rssi: number;
-      snr: number;
+      snr?: number;
-      location: {
+      location?: {
         latitude: number;
         longitude: number;
-        altitude: number;
+        altitude?: number;
         source?: string;
       };
-      uplink_token: string;
+      uplink_token?: string;
       channel_index?: number;
-      received_at: string;
+      received_at?: string;
     }[];
     settings: {
       data_rate: {
         lora: {
           bandwidth: number;
           spreading_factor: number;
-          coding_rate: string;
+          coding_rate?: string;
         };
       };
       frequency: string;
       timestamp?: number;
       time?: Date;
     };
-    received_at: Date;
+    received_at?: Date;
     confirmed?: boolean;
-    consumed_airtime: string;
+    consumed_airtime?: string;
-    version_ids: {
+    version_ids?: {
       brand_id: string;
       model_id: string;
       hardware_version: string;
       firmware_version: string;
       band_id: string;
     };
-    network_ids: {
+    network_ids?: {
       net_id: string;
       ns_id: string;
       tenant_id: string;

server/src/models/wifiLocation.ts

@@ -0,0 +1,52 @@
+import { DataTypes, Model } from "sequelize";
+import { sequelize } from "../database/database";
+
+export class WifiLocation extends Model {
+  public mac!: string;
+  public latitude!: number;
+  public longitude!: number;
+  public request_limit_exceeded!: boolean;
+  public location_not_resolvable!: boolean;
+  public created_at_utc!: Date;
+  public updated_at_utc!: Date;
+}
+
+WifiLocation.init(
+  {
+    mac: {
+      type: DataTypes.STRING,
+      primaryKey: true,
+      allowNull: false,
+    },
+    latitude: {
+      type: DataTypes.NUMBER,
+    },
+    longitude: {
+      type: DataTypes.NUMBER,
+    },
+    request_limit_exceeded: {
+      type: DataTypes.BOOLEAN,
+      defaultValue: false,
+    },
+    location_not_resolvable: {
+      type: DataTypes.BOOLEAN,
+      defaultValue: false,
+    },
+    created_at_utc: {
+      type: DataTypes.DATE,
+      defaultValue: DataTypes.NOW,
+      allowNull: false,
+    },
+    updated_at_utc: {
+      type: DataTypes.DATE,
+      defaultValue: DataTypes.NOW,
+      allowNull: false,
+    },
+  },
+  {
+    sequelize,
+    modelName: "WifiLocation",
+    tableName: "wifi_location",
+    timestamps: false,
+  }
+);

server/src/models/wifiLocationHistory.ts

@@ -0,0 +1,50 @@
+import { DataTypes, Model } from "sequelize";
+import { sequelize } from "../database/database";
+
+export class WifiLocationHistory extends Model {
+  public wifi_location_history_id!: string;
+  public mac!: string;
+  public latitude!: number;
+  public longitude!: number;
+  public created_at_utc!: Date;
+  public updated_at_utc!: Date;
+}
+
+WifiLocationHistory.init(
+  {
+    wifi_location_history_id: {
+      type: DataTypes.UUID,
+      defaultValue: DataTypes.UUIDV4,
+      primaryKey: true,
+      allowNull: false,
+    },
+    mac: {
+      type: DataTypes.STRING,
+      allowNull: false,
+    },
+    latitude: {
+      type: DataTypes.NUMBER,
+      allowNull: false,
+    },
+    longitude: {
+      type: DataTypes.NUMBER,
+      allowNull: false,
+    },
+    created_at_utc: {
+      type: DataTypes.DATE,
+      defaultValue: DataTypes.NOW,
+      allowNull: false,
+    },
+    updated_at_utc: {
+      type: DataTypes.DATE,
+      defaultValue: DataTypes.NOW,
+      allowNull: false,
+    },
+  },
+  {
+    sequelize,
+    modelName: "WifiLocationHistory",
+    tableName: "wifi_location_history",
+    timestamps: false,
+  }
+);

server/src/models/wifiScan.ts

@@ -6,8 +6,7 @@ export class WifiScan extends Model {
   public wifi_scan_id!: string;
   public mac!: string;
   public rssi!: number;
-  public latitude!: number;
+  public scanned_at_utc!: Date;
-  public longitude!: number;
   public created_at_utc!: Date;
   public updated_at_utc!: Date;
 }
@@ -30,15 +29,12 @@ WifiScan.init(
     },
     rssi: {
       type: DataTypes.NUMBER,
-      allowNull: true,
+      allowNull: false,
     },
-    latitude: {
+    scanned_at_utc: {
-      type: DataTypes.NUMBER,
+      type: DataTypes.DATE,
-      allowNull: true,
+      defaultValue: DataTypes.NOW,
-    },
+      allowNull: false,
-    longitude: {
-      type: DataTypes.NUMBER,
-      allowNull: true,
     },
     created_at_utc: {
       type: DataTypes.DATE,

server/src/proxy/wigle.ts

@@ -0,0 +1,71 @@
+interface WigleApiResonse {
+  response?: WifiLocationResponse,
+  status_code: number,
+}
+
+interface WifiLocationResponse {
+  success: boolean;
+  totalResults: number;
+  first: number;
+  last: number;
+  resultCount: number;
+  results: Result[];
+  searchAfter: string;
+  search_after: number;
+}
+
+interface Result {
+  trilat: number;
+  trilong: number;
+  ssid: string;
+  qos: number;
+  transid: string;
+  firsttime: string;
+  lasttime: string;
+  lastupdt: string;
+  netid: string;
+  name?: string;
+  type: string;
+  comment?: string;
+  wep: string;
+  bcninterval: number;
+  freenet: string;
+  dhcp: string;
+  paynet: string;
+  userfound: boolean;
+  channel: number;
+  rcois: string;
+  encryption: string;
+  country: string;
+  region: string;
+  road: string;
+  city?: string;
+  housenumber?: string;
+  postalcode: string;
+}
+
+export const getLocationForWifi = async (
+  mac: string
+): Promise<WigleApiResonse | undefined> => {
+  try {
+    const url = `${process.env.WIGLE_BASE_URL!}${process.env
+      .WIGLE_NETWORK_SEARCH!}?netid=${encodeURIComponent(mac)}`;
+
+    const response = await fetch(url, {
+      method: "GET",
+      headers: {
+        "Content-Type": "application/json",
+        Authorization: `Basic ${process.env.WIGLE_TOKEN}`,
+      },
+    });
+
+    if (response.ok) {
+      return { status_code: response.status, response: await response.json() };
+    }
+    console.log(response.status);
+    return { status_code: response.status };
+  } catch (error) {
+    console.error("Error during call of API wigle.net:", error);
+    return undefined;
+  }
+};

server/src/repositories/locationRepository.ts

@@ -1,10 +1,11 @@
+import { Attributes, FindOptions } from "sequelize";
 import { injectable } from "tsyringe";
 import { Location } from "../models/location";
 
 @injectable()
 export class LocationRepository {
-  public async findAll() {
+  public async findAll(options?: FindOptions<Attributes<Location>>) {
-    return await Location.findAll();
+    return await Location.findAll(options);
   }
 
   public async findById(id: string) {

server/src/repositories/wifiLocationHistoryRepository.ts

@@ -0,0 +1,26 @@
+import { Attributes, FindOptions } from "sequelize";
+import { injectable } from "tsyringe";
+import { WifiLocationHistory } from "../models/wifiLocationHistory";
+
+@injectable()
+export class WifiLocationHistoryRepository {
+  public async findAll() {
+    return await WifiLocationHistory.findAll();
+  }
+
+  public async findOne(options?: FindOptions<Attributes<WifiLocationHistory>>) {
+    return await WifiLocationHistory.findOne(options);
+  }
+
+  public async findById(id: string) {
+    return await WifiLocationHistory.findByPk(id);
+  }
+
+  public async create(data: Partial<WifiLocationHistory>) {
+    return await WifiLocationHistory.create(data);
+  }
+
+  public async createMany(data: Partial<WifiLocationHistory>[]) {
+    return await WifiLocationHistory.bulkCreate(data);
+  }
+}

server/src/repositories/wifiLocationRepository.ts

@@ -0,0 +1,39 @@
+import { Attributes, FindOptions } from "sequelize";
+import { injectable } from "tsyringe";
+import { WifiLocation } from "../models/wifiLocation";
+
+@injectable()
+export class WifiLocationRepository {
+  public async findAll(options?: FindOptions<Attributes<WifiLocation>>) {
+    return await WifiLocation.findAll(options);
+  }
+
+  public async findById(id: string) {
+    return await WifiLocation.findByPk(id);
+  }
+
+  public async create(data: Partial<WifiLocation>) {
+    return await WifiLocation.create(data);
+  }
+
+  public async createMany(data: Partial<WifiLocation>[]) {
+    return await WifiLocation.bulkCreate(data);
+  }
+
+  public async update(id: string, data: Partial<WifiLocation>) {
+    const wifiScan = await this.findById(id);
+    if (wifiScan) {
+      return await wifiScan.update(data);
+    }
+    return null;
+  }
+
+  public async delete(id: string) {
+    const wifiScan = await this.findById(id);
+    if (wifiScan) {
+      await wifiScan.destroy();
+      return true;
+    }
+    return false;
+  }
+}

server/src/services/locationService.ts

@@ -1,24 +1,101 @@
 import { inject, injectable } from "tsyringe";
+import { Op } from 'sequelize';
 import { Location } from "../models/location";
 import { LocationRepository } from "../repositories/locationRepository";
+import { WifiLocationService } from "./wifiLocationService";
+
+interface CreateLocationParams {
+  lp_ttn_end_device_uplinks_id: string;
+  wifi?: Coordinates;
+  gnss?: Coordinates;
+  ttn_gw?: Coordinates;
+  gnss_timestamp?: Date;
+}
+
+interface CreateLocationTriangulationParams {
+  lp_ttn_end_device_uplinks_id: string;
+  wifi: {
+    mac: string;
+    rssi: number;
+  }[];
+  ttn_gw: LocationSignal[];
+  gnss?: Coordinates;
+  gnss_timestamp?: Date;
+}
+
+interface LocationSignal extends Coordinates {
+  rssi: number;
+}
+
+interface Coordinates {
+  latitude: number;
+  longitude: number;
+}
+
+interface UpdateTtnGatewayReceptionParams {
+  ttn_gateway_reception_id: string;
+  gateway_id?: string;
+  eui?: string;
+  rssi?: number;
+  latitude?: number;
+  longitude?: number;
+  altitude?: number;
+}
 
 @injectable()
 export class LocationService {
   constructor(
     @inject(LocationRepository)
-    private repository: LocationRepository
+    private repository: LocationRepository,
-  ) {}
+    @inject(WifiLocationService)
+    private wifiLocationService: WifiLocationService
+  ) { }
 
   public async getAllLocations() {
     return this.repository.findAll();
   }
 
+  public async getAllGnssLocations() {
+    return this.repository.findAll({
+      where: {
+        gnss_latitude: { [Op.ne]: null },
+        gnss_longitude: { [Op.ne]: null }
+      }
+    });
+  }
+
   public async getLocationById(id: string) {
     return this.repository.findById(id);
   }
 
-  public async createLocation(data: Partial<Location>) {
+  public async createLocation(data: CreateLocationParams) {
-    return this.repository.create(data);
+    return this.repository.create({
+      lp_ttn_end_device_uplinks_id: data.lp_ttn_end_device_uplinks_id,
+      wifi_latitude: data.wifi?.latitude,
+      wifi_longitude: data.wifi?.longitude,
+      ttn_gw_latitude: data.ttn_gw?.latitude,
+      ttn_gw_longitude: data.ttn_gw?.longitude,
+      gnss_latitude: data.gnss?.latitude,
+      gnss_longitude: data.gnss?.longitude,
+      gnss_location_at_utc: data.gnss_timestamp,
+    });
+  }
+
+  public async createLocationFromTriangulation(
+    data: CreateLocationTriangulationParams
+  ) {
+    const wifi_location = this.calculateVirtualLocation(
+      await this.enrichWifiObjectWithLocation(data.wifi)
+    );
+    const gateway_location = this.calculateVirtualLocation(data.ttn_gw);
+
+    return this.createLocation({
+      lp_ttn_end_device_uplinks_id: data.lp_ttn_end_device_uplinks_id,
+      wifi: wifi_location,
+      ttn_gw: gateway_location,
+      gnss: data.gnss,
+      gnss_timestamp: data.gnss_timestamp,
+    });
   }
 
   public async updateLocation(id: string, data: Partial<Location>) {
@@ -28,4 +105,48 @@ export class LocationService {
   public async deleteLocation(id: string) {
     return this.repository.delete(id);
   }
+
+  private async enrichWifiObjectWithLocation(
+    wifis: CreateLocationTriangulationParams["wifi"]
+  ) {
+    const enrichedWifi = await Promise.all(
+      wifis.map(async (wifi) => {
+        const location = await this.wifiLocationService.getWifiLocationByMac(
+          wifi.mac
+        );
+
+        return location?.latitude !== undefined &&
+          location.longitude !== undefined
+          ? {
+            rssi: wifi.rssi,
+            latitude: location.latitude,
+            longitude: location.longitude,
+          }
+          : null;
+      })
+    );
+
+    return enrichedWifi.filter((wifi) => wifi !== null);
+  }
+
+  private calculateVirtualLocation(locations: LocationSignal[]) {
+    if (locations.length === 0) return undefined;
+
+    const { totalWeight, weightedLatitude, weightedLongitude } =
+      locations.reduce(
+        (acc, { latitude, longitude, rssi }) => {
+          const weight = 1 / Math.abs(rssi);
+          acc.totalWeight += weight;
+          acc.weightedLatitude += latitude * weight;
+          acc.weightedLongitude += longitude * weight;
+          return acc;
+        },
+        { totalWeight: 0, weightedLatitude: 0, weightedLongitude: 0 }
+      );
+
+    return {
+      latitude: weightedLatitude / totalWeight,
+      longitude: weightedLongitude / totalWeight,
+    };
+  }
 }

server/src/services/metricsService.ts

@@ -0,0 +1,70 @@
+import {injectable } from "tsyringe";
+import { Counter, Gauge, collectDefaultMetrics} from "prom-client";
+
+// Collect default system metrics (e.g., CPU, memory usage)
+const prefix = 'locationhub_';
+collectDefaultMetrics({ prefix });
+
+@injectable()
+export class MetricsService {
+  constructor(
+  ) { }
+
+  public getRequestCounter() {
+    const requestCounter = new Counter({
+      name: `${prefix}http_requests_total`,
+      help: "Total number of HTTP requests",
+      labelNames: ["method", "route", "status"],
+    });
+    return requestCounter;
+  }
+
+  public getLocationsTotal() {
+    const locationsTotal = new Gauge({
+      name: `${prefix}locations_total`,
+      help: "Total number of location entries in database",
+      labelNames: ["database"],
+    });
+    return locationsTotal;
+  }
+  public getGnssLocationsTotal() {
+    const gnssLocationsTotal = new Gauge({
+      name: `${prefix}gnss_locations_total`,
+      help: "Total number of location entries with GNSS in database",
+      labelNames: ["database"],
+    });
+    return gnssLocationsTotal;
+  }
+  public getWifiLocationTotal() {
+    const wifiLocationTotal = new Gauge({
+      name: `${prefix}wifi_locations_total`,
+      help: "Total number of wifi location entries in database",
+      labelNames: ["database"],
+    });
+    return wifiLocationTotal;
+  }
+  public getWifiLocationResolvable() {
+    const wifiLocationNotResolvable = new Gauge({
+      name: `${prefix}wifi_locations_not_resolvable`,
+      help: "Unresolved number of wifi location entries in database",
+      labelNames: ["database"],
+    });
+    return wifiLocationNotResolvable;
+  }
+  public getLWifiLocationRequestLimitExceeded() {
+    const wifiLocationRequestLimitExceeded = new Gauge({
+      name: `${prefix}wifi_locations_request_limit_exceeded`,
+      help: "Unresolved number of wifi location because request limit exceeded entries in database",
+      labelNames: ["database"],
+    });
+    return wifiLocationRequestLimitExceeded;
+  }
+  public getTnnGatewayReceptions() {
+    const ttnGatewayReceptions = new Gauge({
+      name: `${prefix}ttn_gateway_receptions`,
+      help: "Total number of TTN Gateway receptions entries in database",
+      labelNames: ["database"],
+    });
+    return ttnGatewayReceptions;
+  }
+}

server/src/services/ttnGatewayReceptionService.ts

@@ -1,7 +1,26 @@
 import { inject, injectable } from "tsyringe";
-import { TtnGatewayReception } from "../models/ttnGatewayReception";
 import { TtnGatewayReceptionRepository } from "../repositories/ttnGatewayReceptionRepository";
 
+interface CreateTtnGatewayReceptionParams {
+  lp_ttn_end_device_uplinks_id: string;
+  gateway_id: string;
+  eui?: string;
+  rssi?: number;
+  latitude?: number;
+  longitude?: number;
+  altitude?: number;
+}
+
+interface UpdateTtnGatewayReceptionParams {
+  ttn_gateway_reception_id: string;
+  gateway_id?: string;
+  eui?: string;
+  rssi?: number;
+  latitude?: number;
+  longitude?: number;
+  altitude?: number;
+}
+
 @injectable()
 export class TtnGatewayReceptionService {
   constructor(
@@ -17,19 +36,24 @@ export class TtnGatewayReceptionService {
     return this.repository.findById(id);
   }
 
-  public async createGatewayReception(data: Partial<TtnGatewayReception>) {
+  public async createTtnGatewayReception(
-    return this.repository.create(data);
+    data: CreateTtnGatewayReceptionParams
-  }
-
-  public async createGatewayReceptions(data: Partial<TtnGatewayReception>[]) {
-    return this.repository.createMany(data);
-  }
-
-  public async updateGatewayReception(
-    id: string,
-    data: Partial<TtnGatewayReception>
   ) {
-    return this.repository.update(id, data);
+    if (data.latitude !== undefined && data.longitude !== undefined)
+      return this.repository.create(data);
+  }
+
+  public async filterAndInsertGatewayReception(
+    data: CreateTtnGatewayReceptionParams[]
+  ) {
+    const result = await Promise.all(
+      data.map(async (gateway) => await this.createTtnGatewayReception(gateway))
+    );
+    return result.filter((gateway) => gateway !== undefined);
+  }
+
+  public async updateGatewayReception(data: UpdateTtnGatewayReceptionParams) {
+    return this.repository.update(data.ttn_gateway_reception_id, data);
   }
 
   public async deleteGatewayReception(id: string) {

server/src/services/wifiLocationHistoryService.ts

@@ -0,0 +1,51 @@
+import { inject, injectable } from "tsyringe";
+import { WifiLocationHistoryRepository } from "../repositories/wifiLocationHistoryRepository";
+
+interface CreateWifiLocationHistoryParams {
+  mac: string;
+  latitude: number;
+  longitude: number;
+}
+
+interface UpdateWifiLocationHistoryParams {
+  mac: string;
+  latitude: number;
+  longitude: number;
+}
+
+@injectable()
+export class WifiLocationHistoryService {
+  constructor(
+    @inject(WifiLocationHistoryRepository)
+    private repository: WifiLocationHistoryRepository
+  ) {}
+
+  public async getAllWifiLocationHistories() {
+    return this.repository.findAll();
+  }
+
+  public async getWifiLocationHistoryById(id: string) {
+    return this.repository.findById(id);
+  }
+
+  public async createWifiLocationHistory(
+    data: CreateWifiLocationHistoryParams
+  ) {
+    const existingEntry = await this.repository.findOne({
+      where: {
+        mac: data.mac,
+      },
+      order: [["updated_at_utc", "DESC"]],
+    });
+
+    if (
+      !existingEntry ||
+      existingEntry.latitude !== data.latitude ||
+      existingEntry.longitude !== data.longitude
+    ) {
+      return await this.repository.create(data);
+    }
+
+    return existingEntry;
+  }
+}

server/src/services/wifiLocationService.ts

@@ -0,0 +1,92 @@
+import { Op } from "sequelize";
+import { inject, injectable } from "tsyringe";
+import { getLocationForWifi } from "../proxy/wigle";
+import { WifiLocationRepository } from "../repositories/wifiLocationRepository";
+import { WifiLocationHistoryService } from "./wifiLocationHistoryService";
+import { StatusCodes } from "http-status-codes";
+
+interface UpdateWifiLocationParams {
+  mac: string;
+  latitude: number;
+  longitude: number;
+}
+
+@injectable()
+export class WifiLocationService {
+  constructor(
+    @inject(WifiLocationRepository) private repository: WifiLocationRepository,
+    @inject(WifiLocationHistoryService)
+    private wifiLocationHistory: WifiLocationHistoryService
+  ) { }
+
+  public async getAllWifiLocations() {
+    return this.repository.findAll();
+  }
+
+  public async getAllWifiLocationsByAddresses(macAddresses: string[]) {
+    return this.repository.findAll({
+      where: { mac: { [Op.in]: macAddresses } },
+    });
+  }
+
+  public async getAllWifiLocationsByNotResolvable() {
+    return this.repository.findAll({
+      where: { location_not_resolvable: true },
+    });
+  }
+
+  public async getAllWifiLocationsByRequestLimitExceeded() {
+    return this.repository.findAll({
+      where: { request_limit_exceeded: true },
+    });
+  }
+
+  public async getWifiLocationByMac(mac: string) {
+    let wifiLocation = await this.repository.findById(mac);
+
+    if (wifiLocation) return wifiLocation;
+
+    const apiResponse = await getLocationForWifi(mac);
+
+    if (apiResponse == undefined) {
+      await this.repository.create({
+        mac,
+      });
+      return undefined;
+    }
+
+    if (apiResponse.response == undefined) {
+      const request_limit_exceeded = apiResponse.status_code === StatusCodes.TOO_MANY_REQUESTS;
+      await this.repository.create({
+        mac,
+        request_limit_exceeded,
+      });
+      return undefined;
+    }
+
+    if (apiResponse.response.totalResults == 0) {
+      await this.repository.create({ mac, location_not_resolvable: true });
+      return undefined;
+    }
+
+    wifiLocation = await this.repository.create({
+      mac,
+      latitude: apiResponse.response.results[0].trilat,
+      longitude: apiResponse.response.results[0].trilong,
+    });
+
+    await this.wifiLocationHistory.createWifiLocationHistory(
+      wifiLocation.dataValues
+    );
+
+    return wifiLocation;
+  }
+
+  public async updateWifiLocation(data: UpdateWifiLocationParams) {
+    return this.repository.update(data.mac, data);
+  }
+
+  public async deleteWifiLocation(id: string) {
+    return this.repository.delete(id);
+  }
+}

server/src/services/wifiScanService.ts

@@ -1,7 +1,20 @@
 import { inject, injectable } from "tsyringe";
-import { WifiScan } from "../models/wifiScan";
 import { WifiScanRepository } from "../repositories/wifiScanRepository";
 
+interface CreateWifiScanParams {
+  lp_ttn_end_device_uplinks_id: string;
+  mac: string;
+  rssi: number;
+  scanned_at_utc?: Date;
+}
+
+interface UpdateWifiScanParams {
+  wifi_scan_id: string;
+  mac?: string;
+  rssi?: number;
+  scanned_at_utc?: Date;
+}
+
 @injectable()
 export class WifiScanService {
   constructor(
@@ -16,16 +29,16 @@ export class WifiScanService {
     return this.repository.findById(id);
   }
 
-  public async createWifiScan(data: Partial<WifiScan>) {
+  public async createWifiScan(data: CreateWifiScanParams) {
-    return this.repository.create(data);
+      return this.repository.create(data);
   }
 
-  public async createWifiScans(data: Partial<WifiScan>[]) {
+  public async createWifiScans(data: CreateWifiScanParams[]) {
-    return this.repository.createMany(data);
+    return await this.repository.createMany(data);
   }
 
-  public async updateWifiScan(id: string, data: Partial<WifiScan>) {
+  public async updateWifiScan(data: UpdateWifiScanParams) {
-    return this.repository.update(id, data);
+    return this.repository.update(data.wifi_scan_id, data);
   }
 
   public async deleteWifiScan(id: string) {

server/src/validation/ttn/ttnMessageValidation.ts

@@ -8,14 +8,14 @@ export const ttnMessageValidator = z.object({
     }),
     dev_eui: z.string(),
     join_eui: z.string(),
-    dev_addr: z.string(),
+    dev_addr: z.string().optional(),
   }),
   correlation_ids: z.array(z.string()),
   received_at: z.string(),
   uplink_message: z.object({
-    session_key_id: z.string(),
+    session_key_id: z.string().optional(),
     f_port: z.number().optional(),
-    f_cnt: z.number(),
+    f_cnt: z.number().optional(),
     frm_payload: z.string().optional(),
     decoded_payload: z
       .object({
@@ -25,8 +25,8 @@ export const ttnMessageValidator = z.object({
             z.object({
               measurementId: z.string(),
               measurementValue: z.union([z.array(z.any()), z.number()]),
-              motionId: z.number(),
+              motionId: z.number().optional(),
-              timestamp: z.number(),
+              timestamp: z.number().optional(),
               type: z.string(),
             })
           )
@@ -41,20 +41,22 @@ export const ttnMessageValidator = z.object({
           gateway_id: z.string(),
           eui: z.string().optional(),
         }),
-        time: z.string(),
+        time: z.string().optional(),
         timestamp: z.number().optional(),
         rssi: z.number(),
         channel_rssi: z.number(),
-        snr: z.number(),
+        snr: z.number().optional(),
-        location: z.object({
+        location: z
-          latitude: z.number(),
+          .object({
-          longitude: z.number(),
+            latitude: z.number(),
-          altitude: z.number(),
+            longitude: z.number(),
-          source: z.string().optional(),
+            altitude: z.number().optional(),
-        }),
+            source: z.string().optional(),
-        uplink_token: z.string(),
+          })
+          .optional(),
+        uplink_token: z.string().optional(),
         channel_index: z.number().optional(),
-        received_at: z.string(),
+        received_at: z.string().optional(),
       })
     ),
     settings: z.object({
@@ -62,29 +64,33 @@ export const ttnMessageValidator = z.object({
         lora: z.object({
           bandwidth: z.number(),
           spreading_factor: z.number(),
-          coding_rate: z.string(),
+          coding_rate: z.string().optional(),
         }),
       }),
       frequency: z.string(),
       timestamp: z.number().optional(),
       time: z.string().optional(),
     }),
-    received_at: z.string(),
+    received_at: z.string().optional(),
     confirmed: z.boolean().optional(),
-    consumed_airtime: z.string(),
+    consumed_airtime: z.string().optional(),
-    version_ids: z.object({
+    version_ids: z
-      brand_id: z.string(),
+      .object({
-      model_id: z.string(),
+        brand_id: z.string(),
-      hardware_version: z.string(),
+        model_id: z.string(),
-      firmware_version: z.string(),
+        hardware_version: z.string(),
-      band_id: z.string(),
+        firmware_version: z.string(),
-    }),
+        band_id: z.string(),
-    network_ids: z.object({
+      })
-      net_id: z.string(),
+      .optional(),
-      ns_id: z.string(),
+    network_ids: z
-      tenant_id: z.string(),
+      .object({
-      cluster_id: z.string(),
+        net_id: z.string(),
-      cluster_address: z.string(),
+        ns_id: z.string(),
-    }),
+        tenant_id: z.string(),
+        cluster_id: z.string(),
+        cluster_address: z.string(),
+      })
+      .optional(),
   }),
 });