Simple data processing
編輯歷史
| 時間 | 作者 | 版本 |
|---|---|---|
| 2016-01-31 05:12 – 05:12 | r728 – r739 | |
顯示 diff（33 行未修改）
MQTT topic unique
cat data.log | awk '{print $1}' | sort | uniq
+
+ Webduino COPY device count
+ cat data.log | grep 'WEBDUINO_COPY' | awk -F '|' '{print $10}' | awk -F '=' '{print $2}' | sort | uniq
（28 行未修改）
|
||
| 2016-01-19 11:48 – 11:48 | r713 – r727 | |
顯示 diff（30 行未修改）
with version information
cat data.log | awk -F '|' '{print $5"="$6}' | awk -F '=' '{print $2" "$4}' | sort | uniq -c
+
+ MQTT topic unique
+ cat data.log | awk '{print $1}' | sort | uniq
（28 行未修改）
|
||
| 2015-11-22 07:57 – 07:57 | r700 – r712 | |
顯示 diff（27 行未修改）
command:
cat data.log | awk -F '|' '{print $6}' | awk -F '=' '{print $2}' | sort | uniq -c
+
+ with version information
+ cat data.log | awk -F '|' '{print $5"="$6}' | awk -F '=' '{print $2" "$4}' | sort | uniq -c
（28 行未修改）
|
||
| 2015-10-16 09:17 – 09:22 | r687 – r699 | |
顯示 diff（23 行未修改）
You can just copy all results into google spreadsheet to get the char
+ Linux CLI - get all device ids
+
+ command:
+ cat data.log | awk -F '|' '{print $6}' | awk -F '=' '{print $2}' | sort | uniq -c
+
Get statistic information
（27 行未修改）
|
||
| 2015-10-01 03:27 – 04:29 | r407 – r686 | |
顯示 diff（23 行未修改）
You can just copy all results into google spreadsheet to get the char
+
+ Get statistic information
+ This example work with R in the command line, and need R be installed
+
+ Refer this for detail
+
+ New this file and named as stat.r, make sure it can be executed.
+ *#! /usr/bin/env Rscript
+ *d<-scan("stdin", quiet=TRUE)
+ *cat(min(d), max(d), median(d), mean(d),sd(d), sep=",")
+ *cat("\n")
+
+ Command
+ *previous cmd | stat.r
+
+ example result
+ *18.3,38.3,29.9,30.14143,1.957374
+
+ Plot sensors data with time
+ Get time data as column 1
+ cat data.log | grep 'app=MAPS' | grep 'device_id=LASS-MAPS-LJ' | awk -F '|' '{print $9}' | awk -F '=' '{print $2}' > col1.tmp
+
+ Get sensor data as column 2
+ cat data.log | grep 'app=MAPS' | grep 'device_id=LASS-MAPS-LJ' | awk -F '|' '{print $11}' | awk -F '=' '{print $2}' | awk -F ',' '{print $12}' > col2.tmp
+
+ combine 2 columns to CSV format
+ pr -mt -s, col1.tmp col2.tmp > cols.csv
+
+ Import this to google spreadsheet and plot it.
|
||
| 2015-09-30 11:22 – 11:22 | r378 – r406 | |
顯示 diff（1 行未修改）
When you get data from server, there are some of simple way to analyse your data
+
+ Linux CLI - download data
+ wget ftp://gpssensor.ddns.net:2121/data.log
Linux CLI - parse data
（15 行未修改）
You can just copy all results into google spreadsheet to get the char
-
- L
|
||
| 2015-09-30 11:22 | r377 | |
顯示 diff（24 行未修改）
|
||
| 2015-09-30 11:08 – 11:22 | r65 – r376 | |
顯示 diff（2 行未修改）
When you get data from server, there are some of simple way to analyse your data
- Linux command line
+ Linux CLI - parse data
+ example data:
+ LASS/Test/MAPS |ver_format=1|fmt_opt=0|app=MAPS|ver_app=0.6.6|device_id=LASS-MAPS-LJ|tick=22411953|date=30/9/15|time=11:9:30|device=LinkItONE|values=1971.00,100.00,1.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,1008.33,27.80,99.90,0.00,0.00,0.00,0.00,0.00,0.00,0.00|gps=$GPGGA,110930.002,2502.4606,N,12136.8711,E,0,0,,133.6,M,15.3,M,,*42
+
+ purpose: filter app=MAPS and device_id=LASS-MAPS-LJ, get the sensor data column 12. and plot it.
- cat data.log | awk -F '|' '{print $11}' | awk -F '=' '{print $2}' | awk -F ',' '{print $12}'
+ command:
+ cat data.log | grep 'app=MAPS' | grep 'device_id=LASS-MAPS-LJ' | awk -F '|' '{print $11}' | awk -F '=' '{print $2}' | awk -F ',' '{print $12}'
- T
+ How it work:
+ *filter app=MAPS
+ *filter device_id=LASS-MAPS-LJ
+ *get the 11th column by using separator '|' => values=1971.00,100.00,1.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,1008.33,27.80,99.90,0.00,0.00,0.00,0.00,0.00,0.00,0.00
+ *get the second column by using separator '=' => 1971.00,100.00,1.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,1008.33,27.80,99.90,0.00,0.00,0.00,0.00,0.00,0.00,0.00
+ *get the 12th column by using separator ',' => 27.80
+
+ You can just copy all results into google spreadsheet to get the char
+
+ L
|
||
| 2015-09-30 11:08 | r64 | |
顯示 diff（10 行未修改）
|
||
| 2015-09-30 11:07 – 11:08 | r3 – r63 | |
顯示 diff Simple data processing
- W
+ When you get data from server, there are some of simple way to analyse your data
+
+ Linux command line
+
+
+ cat data.log | awk -F '|' '{print $11}' | awk -F '=' '{print $2}' | awk -F ',' '{print $12}'
+
+ T
|
||
| 2015-09-30 11:07 | r2 | |
顯示 diff（3 行未修改）
|
||
| 2015-09-30 11:07 | r1 | |
顯示 diff Simple data processing
+
+ W
|
||
| 2015-09-30 11:07 | r0 | |
顯示 diff-
+ Simple data processing
|
||