Uploaded image for project: 'Kaa'
  1. Kaa
  2. KAA-267

Add Generic Flume event format to Flume log appender

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.1
    • Fix Version/s: 0.6.2
    • Component/s: Operations service
    • Labels:
      None

      Description

      As a platform user I want to have a simple, quick, standard way of integrating Kaa data collection system with data warehousing and processing systems that support standard Flume interface. This story is to update existing flume appender to support new generic flume event format in addition to existing one. According to new generic format every data point should be represented as a separate Flume event, including schema definition in the header mapped by key "flume.avro.schema.literal" (no need to fetch data schema from Kaa Control server via REST);

      The Generic Flume event format will offer limited data throughput compared to the existing format that supports data batching.

        Attachments

          Activity

            People

            Assignee:
            ikulikov Igor Kulikov
            Reporter:
            akokhanovskyi Andrew Kokhanovskyi [Administrator]
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Git Integration