azure-eventhub

Different event types are discriminated by some combination of topics/queues and message headers. In Kafka, record streams are still separated out as topics ( https://kafka.apache.org/intro ). In going through the Azure Event Hubs documentation, I do not see any such idea called out. There are partitions, but these are related to consumer parallelism ( https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-what-is-event-hubs#partitions ), according to the documentation. Is the idea that you would one namespace in Azure Event Hubs, which contains an Event Hub for each "topic"? Yes. I

I'm dealing with a high throughput application of EventHub. According to the documentation , in order to achieve very high throughput from a single sender, then client-side batching is required (without exceeding the 256 KB limit per event). Best Practices for performance improvements using Service Bus brokered messaging suggests Client-side batching for achieving performance improvements. It describes client-side batching is available for queue or topic clients, which enables delaying the sending of messages for a certain period of time, then it transmits the messages in a single batch. Is

What is the best way to send data from Event Hubs to Data Lake Store? I am assuming you want to ingest data from EventHubs to Data Lake Store on a regular basis. Like Nava said, you can use Azure Stream Analytics to get data from EventHub into Azure Storage Blobs. Thereafter you can use Azure Data Factory (ADF) to copy data on a scheduled basis from Blobs to Azure Data Lake Store. More details on using ADF are available here: https://azure.microsoft.com/en-us/documentation/articles/data-factory-azure-datalake-connector/ . Hope this helps. == March 17, 2016 update. Support for Azure Data Lake

Is there any sample code in C# for reading the Azure Event Hub Archive files (Avro format)? I am trying to use the Microsoft.Hadoop.Avro library. I dumped the schema out using a java avro tool which produces this: { ""type"":""record"", ""name"":""EventData"", ""namespace"":""Microsoft.ServiceBus.Messaging"", ""fields"":[ {""name"":""SequenceNumber"",""type"":""long""}, {""name"":""Offset"",""type"":""string""}, {""name"":""EnqueuedTimeUtc"",""type"":""string""}, {""name"":""SystemProperties"",""type"":{ ""type"":""map"",""values"":[""long"",""double"",""string"",""bytes""]}}, {""name"":"

I got an exception of Microsoft.ServiceBus.Messaging.LeaseLostException, in my EventHub processor. what does this exception mean? What is the possible root cause of this exception? Following are stack track: at Microsoft.ServiceBus.Messaging.BlobLeaseManager.d__24.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.ServiceBus.Messaging.BlobLeaseManager.d__25

The Azure Event Hubs overview article states the following: The number of partitions is specified at the Event Hub creation time and must be between 8 and 32. Partitions are a data organization mechanism and are more related to the degree of downstream parallelism required in consuming applications than to Event Hubs throughput. This makes the choice of the number of partitions in an Event Hub directly related to the number of concurrent readers you expect to have. After Event Hub creation, the partition count is not changeable; you should consider this number in terms of long-term expected

I'm currently developing an application hosted on Azure that uses Azure Event Hub. Basically I'm sending messages (or should I say, events) to the Event Hub from a Web API, and I have two listeners: a Stream Analytics task for real-time analysis a standard worker role that computes some stuff based on the received events and then stores them into an Azure SQL Database (this is a lambda architecture). I'm currently using the EventProcessorHost library to retrieve my events from the Event Hub inside my worker role. I'm trying to find some best practices about how to use the Event Hub (it is a

I am working on a IoT solution that will save weather data. I have googled for some days now on how to set up the backend. I am going to use Azure IoT Hub for handling communication, but the next step is the problem. I want to store the telemetry to a database. This is where I get confused. Some examples says that I should use Azure BLOB storage or Azure Table storage or Azure SQL. After some years of data collection I want to start creating reports of the data. So the storage needs to be good at working with big data. Next problem I am stuck on is the worker that will receive the D2C and

We have an Azure Function that has an event hub trigger. Is it possible to have a trigger on more than one event hub for the same Azure Function? Ling Toh No, this is not a supported scenario at the moment. From comments (Ling Toh Jan 18 '17 at 18:23): There are no plans to support multiple triggers per Function. You will have to create a Function for each EventHub. If there is common code that may be shared between Functions, you may move them to a helper method that can be called from each Function. 来源： https://stackoverflow.com/questions/41695474/multiple-triggers-for-an-azure-function