scala - Spark Streaming - Create RDD based on delimiter -
i working spark , have stream of messages flowing seperated message-start delimiter,
i create seperate rdd each message ie. each time see delimiter character.
is possible in spark streaming?
edit:
class routeconsumer extends actor actorhelper consumer { def endpointuri = "rabbitmq://host:5672/xchange?queue=hl7" def receive = { case msg: camelmessage => val m = msg.withbodyas[string] store(m.body) } } val ssc = new streamingcontext(sparkconf, seconds(3)) val dstream = ssc.actorstream[string](props(new routeconsumer()), "sparkreceiveractor") val dstreamsplit = dstream.flatmap(_.split("msh|^~\\&"))
thanks
Comments
Post a Comment