state.HDFSBackedStateStoreProvider: The state for version 1780 doesn't exist in loadedMaps

Question

I am using Spark 2.4.0, kafka 2.10. I am currently working on a Spark Structured Streaming window job and running on YARN. it seems that on every batch interval I get the wa