Kafka integration in unity3d throwing Win32Exception error

问题

I am trying to run a code sample of Kafka in unity environment and for this reason, I created a consumer client (Code given below).

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using Confluent.Kafka;
using Confluent.Kafka.Serialization;
using System.Text;

public class KafkaConsumer : MonoBehaviour 
{
    // Use this for initialization
    void Start () 
    {
        /*
         * The consumer application will then pick the messages from the same topic and write them to console output. 
         * The process to create the consumer application is also very simple.
         */
        var config = new Dictionary<string, object>
        {
            { "group.id","JavaInUseGroup" },
            { "bootstrap.servers", "localhost:9092" },
            { "enable.auto.commit", "false" }
        };

        using (var consumer = new Consumer<Null, string>(config, null, new StringDeserializer(Encoding.UTF8)))
        {
            consumer.Subscribe(new string[] { "javainuse-topic" });

            consumer.OnMessage += (_, msg) =>
            {
                //Console.WriteLine($"Topic: {msg.Topic} Partition: {msg.Partition} Offset :{msg.Offset} {msg.Value}");
                Debug.Log($"Topic: {msg.Topic} Partition: {msg.Partition} Offset :{msg.Offset} {msg.Value}");

                consumer.CommitAsync(msg);
            };

            while (true)
            {
                consumer.Poll(100);
            }
        }
    }
}

In order to execute the above code sample I have also added confluent.Kafka dll into my project asset folder. but whenever I run my unity game it throws an error:Win32Exception: The specified module could not be found.

Rethrow as InvalidOperationException: Error while loading librdkafka.dll or its dependencies from Assets/librdkafka.dll. Check the directory exists, if not check your deployment process. You can also load the library and its dependencies by yourself before any call to Confluent.Kafka Confluent.Kafka.Impl.LibRdKafka.Initialize (System.String userSpecifiedPath) (at <700d5bbe3b974ce5aed001c82b789f6a>:0) Confluent.Kafka.Consumer..ctor (System.Collections.Generic.IEnumerable1[T] config) (at <700d5bbe3b974ce5aed001c82b789f6a>:0) Confluent.Kafka.Consumer2[TKey,TValue]..ctor (System.Collections.Generic.IEnumerable1[T] config, Confluent.Kafka.Serialization.IDeserializer1[T] keyDeserializer, Confluent.Kafka.Serialization.IDeserializer`1[T] valueDeserializer) (at <700d5bbe3b974ce5aed001c82b789f6a>:0) KafkaConsumer.Start () (at Assets/KafkaConsumer.cs:26)

As the error states that there is dependancy problem so i have also copied these dll in assets/librdkafka/x64 folder

1. librdkafka

2. librdkafkacpp

3. msvcr120

4. zlib

Now the problem is, my project get stuck whenever i try to play it.

Remember: I have downloaded all these dll through nuget in vs 2017. Then i bring these dll into unity.

回答1:

For future user, here is the process to add Kafka in your Unity3d Project: Actually there is a specific order or folder hierarchy to add dll in your project. (I didn't find any authoritative reference about this if someone found then please share)

1. First paste Confluen.Kafka dll in whatever your folder

2. Then Make librdkafka folder(Make sure it is created next to confluent.kafka dll) and paste your platform related dll in x64 or x86 Folder

Now, you can run my code sample(mentioned in question).

IMP NOTE: After building the player, you have to manually copy the dll files in your Player/Managed/librdkafka folder. You have to create librdkafka folder in managed folder, then paste your dlls.(Again i don't know why it require but if someone found authoritative refrence then share)

回答2:

If you review the source: https://github.com/confluentinc/confluent-kafka-dotnet/blob/master/src/Confluent.Kafka/Impl/LibRdKafka.cs#L323-L374, confluent.kafka will load librdkafka dynamically, which these dlls:

1. libeay32.dll
2. librdkafka.dll
3. librdkafkacpp.dll
4. libzstd.dll
5. msvcp120.dll
6. msvcr120.dll
7. ssleay32.dll
8. zlib.dll

Either needs to be copied to the same folder where Confluent.Kafka.dll stays or create a librdkafka folder like this to the same folder:

\---librdkafka
    +---x64
    |       libeay32.dll
    |       librdkafka.dll
    |       librdkafkacpp.dll
    |       libzstd.dll
    |       msvcp120.dll
    |       msvcr120.dll
    |       ssleay32.dll
    |       zlib.dll
    |
    \---x86
            libeay32.dll
            librdkafka.dll
            librdkafkacpp.dll
            libzstd.dll
            msvcp120.dll
            msvcr120.dll
            ssleay32.dll
            zlib.dll

来源：https://stackoverflow.com/questions/54703187/kafka-integration-in-unity3d-throwing-win32exception-error