问题
I'm using Cygnus to send Orion Context Broker notifications to Cosmos via httpfs.
Where are stored the data sent to Cosmos in the Hive history? What's the name of the table where Cygnus data are stored?
回答1:
The Orion context data persisted by Cygnus in Cosmos is stored in plain text HDFS files. The content of this files, if properly structured, can be loaded into Hive tables which can be queried by using HiveQL, a SQL-like language.
The way the Hive tables are created depends on the Cygnus version you are using:
- Cygnus 0.1: you have to create the Hive external table by yourself. In order to do that:
- Log into the Cosmos Head Node using your SSH credentials.
- Invoke the Hive CLI by typing
hive - Add the following HiveQL sentence:
create external table <table_name> (recvTimeTs bigint, recvTime string, entityId string, entityType string, attrName string, attrType string, attrValue string) row format delimited fields terminated by '|' location '/user/<myusername>/<mydataset>/'; - Please observe all the entities data is stored within the same and unique Hive table. This is possible because all the lines/rows within HDFS files/Hive table refer to an attribute of certain type belonging to an entity identifier or certain type.
- Cygnus 0.2: the above Hive external table is automatically created. The table name is
<myusername>_<mydataset>. As in Cygnus 0.1, all the entities data is stores within the same and unique Hive table. - Cygnus 0.3 or greater: at the moment of writing this response, Cygnus 0.3 has not yet been released, but within such release the Orion data will not be exclusively persisted by adding a new line/row per each attribute, and the possibility to add new lines/rows containing full entity's attribute list is expected. In that case, due to the lines/rows may not have the same dimenssion, it is envioned a Hive table is created per each entity.
来源：https://stackoverflow.com/questions/24204874/how-to-know-the-cygnus-notifications-table-name-in-cosmos