azure-eventhub

Spark streaming for Azure Event hubs

ⅰ亾dé卋堺 提交于 2019-12-12 00:41:24
问题 I tried the given process (https://azure.microsoft.com/en-in/documentation/articles/hdinsight-apache-spark-eventhub-streaming/) step by step. I have just modified the spark receiver code according to my requirement. The spark streaming consumer api when I am spark-submitting its fetching the data from EventHub as DStream[Array[Bytes]] which I am doing a foreachRDD and converting into an RDD[String] . The issue I am facing here is that the statements below the streaming line are not getting

Azure Event Hubs Changing Partition Key

时光总嘲笑我的痴心妄想 提交于 2019-12-11 23:35:39
问题 An application sets the value of EventData.PartitionKey to a new Guid upon start-up. For each new deployment, the Partition Key will therefore change. I understand that Event Hubs leverage a hashing mechanism in order to route messages to specific Partitions. Does regenerating the Partition Key impede, or affect, this mechanism in any detrimental way? I notice from time to time that messages do not appear in the Event Hub (regardless of how much time has passed) after multiple deployments,

Azure Function with Event Hub out binding does not work

末鹿安然 提交于 2019-12-11 15:19:12
问题 I am using Azure Function to send out message to multiple Event Hub outputs (from an input EH). My code is the following: [FunctionName("Gateway")] public static void Run([EventHubTrigger("%INPUT_HUB_NAME%", Connection = "iothubconnection", ConsumerGroup = "functiontest")]EventData[] eventHubMessage, [EventHub("%OUT_HUB_NAME%", Connection = "eventhuboutput")]out EventData outputEventHubMessageHotPath, [EventHub("%OUT_HUB_NAME%", Connection = "eventhuboutput2")]out EventData

How to receive the recent data only in event hub

不打扰是莪最后的温柔 提交于 2019-12-11 14:59:05
问题 In eventhub, I have both "sender" and "receiver" scripts for communication between those two. The issue that I am facing is that it seems that I am receiving a dataset that I sent yesterday plus the one that I just sent together. I am trying to control the data amount by either time period or the number of events. The basic code for sender.py is following: CONSUMER_GROUP = "$default" OFFSET = Offset("-1") PARTITION = "0" total = 0 last_sn = -1 last_offset = "-1" client = EventHubClient

python script which subscribes/listens to Azure Event Hub?

℡╲_俬逩灬. 提交于 2019-12-11 12:13:39
问题 I am interested in listening to an Event Hub from a Python script and executing some code when an inbound message is detected. On Ubuntu, using Python 2.7 with https://pypi.python.org/pypi/python-qpid-proton/0.10 and the example "receiver" script, I attempt to subscribe to a URI that looks like: amqps://<key name>:<key>@<namespace>.servicebus.windows.net/<event hub name>/ConsumerGroups/$Default/Partitions/0 The following error is returned: proton.MessengerException: Cannot subscribe to <...>

Sending AMQP Telemetry to Azure Event Hub not using TLS encryption

ε祈祈猫儿з 提交于 2019-12-11 12:02:37
问题 Is it possible to send AMQP packets/telemetry to Azure Event Hub not using TLS encryption? The reason I want to do this, is to see exactly what a non-encrypted AMQP packet looks like. I have tried the following code (see amqpTransSetting.UseSslStreamSecurity = false , but it still encrypts the communication. var amqpTransSetting = new AmqpTransportSettings(); amqpTransSetting.UseSslStreamSecurity = false; var cs = @"Endpoint=sb://companyname-ns.servicebus.windows.net/;SharedAccessKeyName

How to implement Routes functionality for Twin Change in Azure Iot-hub?

北城以北 提交于 2019-12-11 10:40:36
问题 I have set Routes for update twin and select endpoint as events(default) supported by Iot hub but it is not working. Here is in screenshot Am i doing anything wrong or something is missing ? Any advice or references to other material would be appreciated. 回答1: Your Routes setup is correct. I am guessing the problem is on the consumer side of the Azure IoT Hub events (default endpoint). You can use for test purpose a Device Explorer tool. The following screen snippet shows my example: other

Is there any way to validate events payload schema while using Azure Event Hub API?

主宰稳场 提交于 2019-12-11 10:27:51
问题 I must send events to Azure Event Hub directly using http protocol. How can I validate payload json schema of the event in Azure (same as validating in a web api) before the event ingestion? 来源: https://stackoverflow.com/questions/54789404/is-there-any-way-to-validate-events-payload-schema-while-using-azure-event-hub-a

Subscribe to Azure Cloud Partner Portal

血红的双手。 提交于 2019-12-11 06:16:39
问题 I am trying to automate the offer publishing process on Azure Cloud Partner Portal. I would like to get notified when the offer reaches the preview stage. One way to do this would be to deploy a serverless function which polls the cloudpartner api to check the status of the offer. I was wondering if there was a cleaner solution to this problem though. I have been reading about Event Grid at the following page: https://docs.microsoft.com/en-us/azure/event-grid/overview It seems like there is a

Azure Event Hub timeouts

会有一股神秘感。 提交于 2019-12-10 18:23:08
问题 I occasionally see timeout errors in event hub client when trying to send messages to Azure event hub. It looks like resource limit is reached for client but I'm not sure... Here the code: MessagingFactory messagingFactory = null; EventHubClient hubClient = null; try { messagingFactory = MessagingFactory.CreateFromConnectionString(this.connectionString); hubClient = messagingFactory.CreateEventHubClient(this.configurationManager.EventHubName); var batchIterator = new EventBatchIterator<T>