foreachRDD returning N RDDs when?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP


foreachRDD returning N RDDs when?



There has been some discussion on foreachRDD in Spark Streaming as per a good posting stackoverflow 36421619. None-the-less I feel the answers are not clear enough when reading the prose. So here goes ...



My questions are:



The commonest use case appears to be for persisting to external storage I note. That seems to be the guide for most, output operations. I am somehow missing something.




1 Answer
1



Only with windowed operation, else 1 microbatch = 1 RDD.





Documentation a little confusing.
– thebluephantom
Dec 2 '17 at 10:18






By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Popular posts from this blog

Stripe::AuthenticationError No API key provided. Set your API key using “Stripe.api_key = ”

CRM reporting Extension - SSRS instance is blank

Keycloak server returning user_not_found error when user is already imported with LDAP