data-integration

In a data flow from one table to another, I would like to cast a date. The date leaves the source table as a string in this format: "2009-01-05 00:00:00:000 + 01:00". I tried to convert this to a date using a tConvertType, but that is not allowed apparently. My second option is to cast this string to a date using a formula in a tMap component. At the moment I tried these formulas: - TalendDate.formatDate("yyyy-MM-dd",row3.rafw_dz_begi); - TalendDate.formatDate("yyyy-MM-dd HH:mm:ss",row3.rafw_dz_begi); - return TalendDate.formatDate("yyyy-MM-dd HH:mm:ss",row3.rafw_dz_begi); None of these worked

I am using Kafka and Zookeeper as the main components of my data pipeline, which is processing thousands of requests each second. I am using Samza as the real time data processing tool for small transformations that I need to make on the data. My problem is that one of my consumers (lets say ConsumerA ) consumes several topics from Kafka and processes them. Basically creating a summary of the topics that are digested. I further want to push this data to Kafka as a separate topic but that forms a loop on Kafka and my component. This is what bothers me, is this a desired architecture in Kafka?

I have various CSV files incoming several times per day, storing timeseries data from sensors, which are parts of sensors stations. Each CSV is named after the sensor station and sensor id from which it is coming from, for instance "station1_sensor2.csv". At the moment, data is stored like this : > cat station1_sensor2.csv 2016-05-04 03:02:01.001000+0000;0; 2016-05-04 03:02:01.002000+0000;0.1234; 2016-05-04 03:02:01.003000+0000;0.2345; I have created a Cassandra table to store them and to be able to query them for various identified tasks. The Cassandra table looks like this : cqlsh > CREATE