倒排索引/搜索引擎>> zeebe-csv-exporter>> 返回
项目作者： zeebe-io

项目描述 ：
Zeebe CSV exporter
高级语言： Java
项目地址: git://github.com/zeebe-io/zeebe-csv-exporter.git
创建时间： 2019-02-24T19:03:03Z
项目社区：https://github.com/zeebe-io/zeebe-csv-exporter
开源协议：Apache License 2.0
下载


Zeebe CSV Exporter

A simple aggregating CSV for Zeebe.

Note: This is a work in progress; you’re welcome to contribute code or ideas, but no guarantees are made about the exporter itself.
Use at your own risks.

Usage

  1. Download the latest release from GitHub and copy it into the Zeebe lib
    folder

  2. Add the exporter configuration to the conf/zeebe.cfg.toml to enable the
    exporter

    1. [[exporters]]
    2. id = "csv"
    3. className = "io.zeebe.exporter.CsvExporter"
    5. [exporters.args]
    6. # output path to write csv files
    7. output = "csv/"

Example

The exporter writes a file per partition for workflow instances and jobs.

With the exporter configuration shown above and a broker with 3 partitions the
following files will be created.

  1. csv/
  2. ├── job-0.csv
  3. ├── job-1.csv
  4. ├── job-2.csv
  5. ├── workflow-instance-0.csv
  6. ├── workflow-instance-1.csv
  7. └── workflow-instance-2.csv

The content of the files contains a header line with the field names and then
a line for every finished entity.

  1. $ head -n10 csv/workflow-instance-0.csv
  2. bpmnElementType,bpmnProcessId,completed,created,duration,elementId,endPosition,ended,flowScopeKey,key,partition,startPosition,version,workflowInstanceKey,workflowKey
  3. START_EVENT,process,true,1551051793053,1,StartEvent_1,4295035184,1551051793054,155,156,0,4295034440,1,155,1
  4. START_EVENT,process,true,1551051793073,1,StartEvent_1,4295038368,1551051793074,160,161,0,4295037624,1,160,1
  5. START_EVENT,process,true,1551051793100,1,StartEvent_1,4295043072,1551051793101,166,167,0,4295042328,1,166,1
  6. SERVICE_TASK,process,true,1551051793056,62,ServiceTask_0kt6c5i,4295046760,1551051793118,155,158,0,4295035704,1,155,1
  7. SERVICE_TASK,process,true,1551051793074,50,ServiceTask_0kt6c5i,4295047024,1551051793124,160,163,0,4295038888,1,160,1
  8. SERVICE_TASK,process,true,1551051793103,44,ServiceTask_0kt6c5i,4295051008,1551051793147,166,169,0,4295043592,1,166,1
  9. END_EVENT,process,true,1551051793141,7,EndEvent_0crvjrk,4295051272,1551051793148,155,174,0,4295048872,1,155,1
  10. END_EVENT,process,true,1551051793141,8,EndEvent_0crvjrk,4295051528,1551051793149,160,175,0,4295049128,1,160,1
  11. PROCESS,process,true,1551051793046,108,process,4295053168,1551051793154,-1,155,0,4295033960,1,155,1
  1. $ head -n10 csv/job-0.csv
  2. bpmnProcessId,completed,created,duration,elementId,elementInstanceKey,endPosition,ended,key,partition,startPosition,type,"workflowDefinitionVersion",workflowInstanceKey,workflowKey
  3. process,true,1551051793061,50,ServiceTask_0kt6c5i,158,4295045384,1551051793111,159,0,4295036576,task,1,155,1
  4. process,true,1551051793075,39,ServiceTask_0kt6c5i,163,4295045728,1551051793114,164,0,4295039760,task,1,160,1
  5. process,true,1551051793103,40,ServiceTask_0kt6c5i,169,4295049384,1551051793143,170,0,4295044464,task,1,166,1
  6. process,true,1551051793174,4,ServiceTask_0kt6c5i,185,4295060624,1551051793178,186,0,4295058872,task,1,181,1
  7. process,true,1551051793210,11,ServiceTask_0kt6c5i,199,4295070088,1551051793221,201,0,4295068512,task,1,196,1
  8. process,true,1551051793320,19,ServiceTask_0kt6c5i,225,4295090616,1551051793339,226,0,4295085864,task,1,222,1
  9. process,true,1551051793276,65,ServiceTask_0kt6c5i,215,4295092312,1551051793341,216,0,4295079496,task,1,212,1
  10. process,true,1551051793301,40,ServiceTask_0kt6c5i,220,4295092920,1551051793341,221,0,4295082680,task,1,217,1
  11. process,true,1551051793248,96,ServiceTask_0kt6c5i,210,4295097112,1551051793344,211,0,4295076312,task,1,207,1

Analyse

The CSV files can be imported into a spreadsheet, a SQL database or used as
input to your favorite command line tool to be further analysed.

See the scripts/average-duration.awk example how to process the CSV with
AWK to calculate the count, total and average duration per workflow instance
element or job.

  1. awk -f scripts/average-duration.awk samples/workflow-instance-0.csv

Or use a tool like textql to run SQL queries on-the-fly.

  1. textql -header -pretty -sql 'select elementId, count(duration), sum(duration), avg(duration) from "workflow-instance-0" group by elementId' samples/workflow-instance-0.csv

Get the number of created elements per second grouped by element id

  1. textql -header -pretty -sql 'SELECT elementId, datetime(created/1000, "unixepoch") as timestamp, count() from "workflow-instance-0" GROUP BY elementId, timestamp ORDER BY timestamp' samples/workflow-instance-0.csv

The average number of completed elements per second grouped by element id

  1. textql -header -pretty -sql 'select elementId, avg(c) FROM (SELECT elementId, datetime(ended/1000, "unixepoch") as timestamp, count() as c GROUP BY elementId, timestamp) group by elementId' samples/workflow-instance-0.csv

Code of Conduct

This project adheres to the Contributor Covenant Code of
Conduct. By participating, you are expected to uphold
this code. Please report unacceptable behavior to code-of-conduct@zeebe.io.

License

Apache License, Version 2.0