spring-cloud-dataflow

I'm exploring the the SCDF for orchestrating the existing batch task for my system. I noticed that the SC Stream App Starters already provides some OOTB ready to use streaming applications, which suits my use case of: Extracting xml file from FTP -> the ftp source Customize transformation Upload the transformed xml file to another FTP -> the ftp sink But, my system only requires the above mentioned batch job to be executed at every 10 minutes interval. Implementing it using streams app is fine but it requires the stream app to be running 24-7 (using SCDF with Kubernetes). My concern is that I

I have been looking at using projects built using spring-cloud-task within spring-cloud-dataflow. Having looked at the example projects and the documentation, the indication seems to be that tasks are launched manually through the dashboard or the shell. Does spring-cloud-dataflow provide any way of scheduling task definitions so that they can run for example on a cron schedule? I.e. Can you create a spring-cloud-task app which itself has no knowledge of a schedule, but deploy it to the dataflow server and configure the scheduling there? Among the posts and blogs I have looked at I noticed the

I would like to use streams to handle an HTTP request/response exchange. I didn't see any Spring Cloud Stream App Starters with HTTP sink functionality. Will I need to build a custom sink to handle the response? If so, do I pass the request through my processing pipeline, then use the request in my sink to form the response? I don't think I've misunderstood the use case of Spring Cloud DataFlow and Spring Cloud Stream. Perhaps there are app starters available for this pattern. Spring Cloud Stream/Dataflow is for unidirectional (stream) processing; it is not intended for request/reply

Based on what I've seen, creating a stream in Spring Cloud Dataflow (SCDF) will deploy the underlying applications, bind the communication service (like RabbitMQ), set the Spring Cloud Stream environment variables, and start the applications. This could all be done manually easily using a cf push command. Meanwhile, I've been running into some drawbacks with Spring Cloud Dataflow: SCDF Server is a memory hog on PCF (I have a stream with only 6 applications, and yet I'm needing about 10GB for the server) No flexibility on application naming, memory, instances, etc. (All the things that you