biginsights

RSA premaster secret error when using webHDFS from BigInsights on cloud

由 心已入冬 提交于 2020-02-08 03:17:08
问题 I'm trying to execute the spark+oozie+bluemix liberty example on the OozieWorkflowSparkGroovyBluemixDeploy branch against a BigInsights for Apache Hadoop basic cluster. The error I get when I try to access the application from a browser: There was an unexpected error (type=Internal Server Error, status=500). javax.net.ssl.SSLKeyException: RSA premaster secret error What is causing this issue? 回答1: The issue appears to be due to the webHDFS certificate not being in the liberty truststore. See

How can I use python > 2.6.6 with spark on BigInsights on cloud Enterprise clusters?

由 纵然是瞬间 提交于 2019-12-24 08:23:52
问题 The version of python with BigInsights is currently 2.6.6. How can I use a different version of Python with my spark jobs running on yarn? Note that users of BigInsights on cloud do not have root access. 回答1: Install Anaconda This script installs anaconda python on a BigInsights on cloud 4.2 Enterprise cluster. Note that these instructions do NOT work for Basic clusters because you are only able to login to a shell node and not any other nodes. Ssh into the mastermanager node, then run

Spark Hive reporting pyspark.sql.utils.AnalysisException: u'Table not found: XXX' when run on yarn cluster

由 |▌冷眼眸甩不掉的悲伤 提交于 2019-12-19 11:22:58
问题 I'm attempting to run a pyspark script on BigInsights on Cloud 4.2 Enterprise that accesses a Hive table. First I create the hive table: [biadmin@bi4c-xxxxx-mastermanager ~]$ hive hive> CREATE TABLE pokes (foo INT, bar STRING); OK Time taken: 2.147 seconds hive> LOAD DATA LOCAL INPATH '/usr/iop/4.2.0.0/hive/doc/examples/files/kv1.txt' OVERWRITE INTO TABLE pokes; Loading data to table default.pokes Table default.pokes stats: [numFiles=1, numRows=0, totalSize=5812, rawDataSize=0] OK Time taken:

How to load a flat file(not delimited file) into HBase?

由 老子叫甜甜 提交于 2019-12-12 05:29:09
问题 I am new to hbase and I have a flat file(not delimited file) that I would like to load into a single hbase table. Here is a preview of a row in my file: 0107E07201512310015071C11100747012015123100 I know fo example that from position 1 to 7 it's an id and from position 7 to 15 it's a date.... The problem is how to build a schema that correspond to my file or if there is a way to convert it to a delimited file or read such file using jaql because I'm working with Infosphere BigInsights. Any

hive spark yarn-cluster job fails with: “ClassNotFoundException: org.datanucleus.api.jdo.JDOPersistenceManagerFactory”

由 若如初见. 提交于 2019-12-02 13:32:00
问题 I'm attempting to run a pyspark script on BigInsights on Cloud 4.2 Enterprise that accesses a Hive table. First I create the hive table: [biadmin@bi4c-xxxxx-mastermanager ~]$ hive hive> CREATE TABLE pokes (foo INT, bar STRING); OK Time taken: 2.147 seconds hive> LOAD DATA LOCAL INPATH '/usr/iop/4.2.0.0/hive/doc/examples/files/kv1.txt' OVERWRITE INTO TABLE pokes; Loading data to table default.pokes Table default.pokes stats: [numFiles=1, numRows=0, totalSize=5812, rawDataSize=0] OK Time taken:

hive spark yarn-cluster job fails with: “ClassNotFoundException: org.datanucleus.api.jdo.JDOPersistenceManagerFactory”

由 点点圈 提交于 2019-12-02 07:21:31
I'm attempting to run a pyspark script on BigInsights on Cloud 4.2 Enterprise that accesses a Hive table. First I create the hive table: [biadmin@bi4c-xxxxx-mastermanager ~]$ hive hive> CREATE TABLE pokes (foo INT, bar STRING); OK Time taken: 2.147 seconds hive> LOAD DATA LOCAL INPATH '/usr/iop/4.2.0.0/hive/doc/examples/files/kv1.txt' OVERWRITE INTO TABLE pokes; Loading data to table default.pokes Table default.pokes stats: [numFiles=1, numRows=0, totalSize=5812, rawDataSize=0] OK Time taken: 0.49 seconds hive> Then I create a simple pyspark script: [biadmin@bi4c-xxxxxx-mastermanager ~]$ cat

Spark Hive reporting ClassNotFoundException: com.ibm.biginsights.bigsql.sync.BIEventListener

由 一曲冷凌霜 提交于 2019-12-02 06:42:13
问题 I'm attempting to run a pyspark script on BigInsights on Cloud 4.2 Enterprise that accesses a Hive table. First I create the hive table: [biadmin@bi4c-xxxxx-mastermanager ~]$ hive hive> CREATE TABLE pokes (foo INT, bar STRING); OK Time taken: 2.147 seconds hive> LOAD DATA LOCAL INPATH '/usr/iop/4.2.0.0/hive/doc/examples/files/kv1.txt' OVERWRITE INTO TABLE pokes; Loading data to table default.pokes Table default.pokes stats: [numFiles=1, numRows=0, totalSize=5812, rawDataSize=0] OK Time taken:

Spark Hive reporting pyspark.sql.utils.AnalysisException: u'Table not found: XXX' when run on yarn cluster

由 送分小仙女□ 提交于 2019-12-01 11:41:40
I'm attempting to run a pyspark script on BigInsights on Cloud 4.2 Enterprise that accesses a Hive table. First I create the hive table: [biadmin@bi4c-xxxxx-mastermanager ~]$ hive hive> CREATE TABLE pokes (foo INT, bar STRING); OK Time taken: 2.147 seconds hive> LOAD DATA LOCAL INPATH '/usr/iop/4.2.0.0/hive/doc/examples/files/kv1.txt' OVERWRITE INTO TABLE pokes; Loading data to table default.pokes Table default.pokes stats: [numFiles=1, numRows=0, totalSize=5812, rawDataSize=0] OK Time taken: 0.49 seconds hive> Then I create a simple pyspark script: [biadmin@bi4c-xxxxxx-mastermanager ~]$ cat
订阅 biginsights