Cygnus 0.8.2 doesn't work -
i have installed cygnus 0.8.2 in vm centos-6.5-x64. in config file agent.conf change following:
cygnusagent.sinks.hdfs-sink.oauth2_token = xxxxxx cygnusagent.sinks.hdfs-sink.hdfs_username = myuser i run cygnus command:
/usr/cygnus/bin/cygnus-flume-ng agent --conf /usr/cygnus/conf/ -f /usr/cygnus/conf/agent.conf -n cygnusagent -dflume.root.logger=info,console but when send xml message occurs error:
the last packet sent server 0 milliseconds ago. driver has not received packets server.) 2015-07-23 14:46:02,133 (sinkrunner-pollingrunner-defaultsinkprocessor) [warn - com.telefonica.iot.cygnus.sinks.orionsink.process(orionsink.java:163)] event ttl has expired, no more re-injected in channel (id=617320308, ttl=0) 2015-07-23 14:46:02,133 (sinkrunner-pollingrunner-defaultsinkprocessor) [info - com.telefonica.iot.cygnus.sinks.orionsink.process(orionsink.java:193)] finishing transaction (1437651766-108-0000000000) 2015-07-23 14:46:02,637 (sinkrunner-pollingrunner-defaultsinkprocessor) [info - com.telefonica.iot.cygnus.sinks.orionsink.process(orionsink.java:128)] event got channel (id=617320308, headers={timestamp=1437651946136, content-type=application/xml, transactionid=1437651766-108-0000000000, fiware-service=def_serv, fiware-servicepath=def_servpath, ttl=0, destination=sensorreading1_sensorreading}, bodylength=891) 2015-07-23 14:46:02,643 (sinkrunner-pollingrunner-defaultsinkprocessor) [info - com.telefonica.iot.cygnus.sinks.orionhdfssink.persist(orionhdfssink.java:356)] [hdfs-sink] persisting data @ orionhdfssink. hdfs file (def_serv/def_servpath/sensorreading1_sensorreading/sensorreading1_sensorreading.txt), data ({"recvtime":"2015-07-23t11:45:46.136z","nodeid":"1", "nodeid_md":[],"sensorid":"1", "sensorid_md":[],"systemid":"1", "systemid_md":[],"value":"-990.6", "value_md":[]}) 2015-07-23 14:46:02,644 (sinkrunner-pollingrunner-defaultsinkprocessor) [error - com.telefonica.iot.cygnus.sinks.orionsink.process(orionsink.java:143)] persistence error (the /user/cristina.albaladejo/def_serv/def_servpath/sensorreading1_sensorreading directory not created in hdfs. httpfs response: 503 service unavailable) 2015-07-23 14:46:02,644 (sinkrunner-pollingrunner-defaultsinkprocessor) [warn - com.telefonica.iot.cygnus.sinks.orionsink.process(orionsink.java:163)] event ttl has expired, no more re-injected in channel (id=617320308, ttl=0) 2015-07-23 14:46:02,644 (sinkrunner-pollingrunner-defaultsinkprocessor) [info - com.telefonica.iot.cygnus.sinks.orionsink.process(orionsink.java:193)] finishing transaction (1437651766-108-0000000000) then, data not stored in hdfs... how can solve this?
the complete config is:
cygnusagent.sources = http-source cygnusagent.sinks = hdfs-sink mysql-sink ckan-sink cygnusagent.channels = hdfs-channel mysql-channel ckan-channel cygnusagent.sources.http-source.channels = hdfs-channel mysql-channel ckan-channel cygnusagent.sources.http-source.type = org.apache.flume.source.http.httpsource cygnusagent.sources.http-source.port = 5050 cygnusagent.sources.http-source.handler = com.telefonica.iot.cygnus.handlers.orionresthandler cygnusagent.sources.http-source.handler.notification_target = /notify cygnusagent.sources.http-source.handler.default_service = def_serv cygnusagent.sources.http-source.handler.default_service_path = def_servpath cygnusagent.sources.http-source.handler.events_ttl = 10 cygnusagent.sources.http-source.interceptors = ts gi cygnusagent.sources.http-source.interceptors.ts.type = timestamp cygnusagent.sources.http-source.interceptors.gi.type = com.telefonica.iot.cygnus.interceptors.groupinginterceptor$builder cygnusagent.sources.http-source.interceptors.gi.grouping_rules_conf_file = /usr/cygnus/conf/grouping_rules.conf # orionhdfssink configuration cygnusagent.sinks.hdfs-sink.channel = hdfs-channel cygnusagent.sinks.hdfs-sink.type = com.telefonica.iot.cygnus.sinks.orionhdfssink cygnusagent.sinks.hdfs-sink.hdfs_host = x1.y1.z1.w1,x2.y2.z2.w2 cygnusagent.sinks.hdfs-sink.hdfs_port = 14000 cygnusagent.sinks.hdfs-sink.hdfs_username = cristina.albaladejo cygnusagent.sinks.hdfs-sink.oauth2_token = mytoken cygnusagent.sinks.hdfs-sink.attr_persistence = column cygnusagent.sinks.hdfs-sink.hive_host = x.y.z.w cygnusagent.sinks.hdfs-sink.hive_port = 10000 cygnusagent.sinks.hdfs-sink.krb5_auth = false cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_user = krb5_username cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_password = xxxxxxxxxxxxx cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_login_conf_file = /usr/cygnus/conf/krb5_login.conf cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_conf_file = /usr/cygnus/conf/krb5.conf cygnusagent.channels.hdfs-channel.type = memory cygnusagent.channels.hdfs-channel.capacity = 1000 cygnusagent.channels.hdfs-channel.transactioncapacity = 100
it seems have not configured following properties (putting required values):
cygnusagent.sinks.hdfs-sink.hdfs_host = cosmos.lab.fiware.org cygnusagent.sinks.hdfs-sink.hive_host = cosmos.lab.fiware.org in addition, since configuring hdfs persistence, avoid reference mysql-sink , ckan-sink, i.e.:
cygnusagent.sinks = hdfs-sink cygnusagent.channels = hdfs-channel cygnusagent.sources.http-source.channels = hdfs-channel try proposed changes , let me know if works.
Comments
Post a Comment