问题
I am using Solr 4.6.0 and trying to import my data using CachedSqlEntityProcessor, but somehow I end up getting a ClassCastException.
Schema
<fields>
<field name="_version_" type="long" indexed="true" stored="true"/>
<field name="id" type="int" indexed="true" stored="true" required="true" multiValued="false" />
<field name="conference" type="string" indexed="true" stored="true" />
<field name="year" type="int" indexed="true" stored="true" />
<field name="doi" type="string" indexed="false" stored="true" />
<field name="text" type="text_en_shingling" indexed="true" stored="true" />
</fields>
Data config
<dataConfig>
<dataSource name="mysql" type="JdbcDataSource"
driver="com.mysql.jdbc.Driver"
url="..."
batchSize="-1"
user="..." password="..." />
<document name="publications">
<entity name="publication" query="
SELECT
sm_publications.id AS p_id,
sm_publications.year AS p_year,
sm_publications.doi AS p_doi,
CONCAT(sm_publications.title, ' ', sm_publications.abstract) AS p_text,
sm_publications.conference_id
FROM sm_publications
WHERE '${dataimporter.request.clean}' != 'false' OR
modified > '${dataimporter.last_index_time}'">
<field column="p_id" name="id" />
<field column="p_text" name="text" />
<field column="p_year" name="year" />
<field column="p_doi" name="doi" />
<entity name="conference" query="SELECT id AS CID,full_name FROM sm_conferences"
processor="CachedSqlEntityProcessor" cacheKey="CID" cacheLookup="publication.conference_id">
<field column="full_name" name="conference" />
</entity>
</entity>
</document>
</dataConfig>
Error message
Full Import failed:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:270)
at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:411)
at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:476)
at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:179)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:710)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:413)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:410)
at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:323)
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:231)
... 33 more
Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String
at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:63)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:246)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:469)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:495)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:408)
... 35 more
Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String
at java.lang.String.compareTo(Unknown Source)
at java.util.TreeMap.getEntry(Unknown Source)
at java.util.TreeMap.get(Unknown Source)
at org.apache.solr.handler.dataimport.SortedMapBackedCache.iterator(SortedMapBackedCache.java:147)
at org.apache.solr.handler.dataimport.DIHCacheSupport.getIdCacheData(DIHCacheSupport.java:179)
at org.apache.solr.handler.dataimport.DIHCacheSupport.getCacheData(DIHCacheSupport.java:145)
at org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:129)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:75)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:243)
... 38 more
Table structure
回答1:
CachedSqlEntityProcessor relies on DIHCacheSupport(Map<String,Object> getIdCacheData(...) ) that is not adapted to work with keys of type Integer, which is expected behavior for any cache. (Object key = context.resolve(cacheForeignKey);, this key should really be of type String)
SELECT CAST(id as CHAR(32)) AS CID,full_name FROM sm_conferences
回答2:
This is a regression coming out of the introduction of pluggable cache support in https://issues.apache.org/jira/browse/SOLR-2382 and the workaround (working for me) is to cast to strings in all columns that you will use as either keys or values. In PostgreSQL cast syntax:
<entity name="par" dataSource="d" query="SELECT id, xyz, child_id::text FROM par;">
<entity name="child" dataSource="d" query="SELECT id::text, abc FROM child;"
processor="CachedSqlEntityProcessor"
where="id=par.child_id"/>
</entity>
(and in other DBs I guess you need CAST(id AS VARCHAR(10) or such instead of id::text).
回答3:
This is a matter of case. You need to write the key in the cacheLookup attribute in upper-case. If you use
cacheLookup="publication.CONFERENCE_ID"
it will work.
All columns you gather for the entity publication are gathered within a Map internally. The keys to that map are uppercased. When you write cacheLookup="publication.CONFERENCE_ID" that short part after the equals sign publication.CONFERENCE_ID is a short-cut to that map. Apparently a String#toUpper is missing at some point in between.
I have added a test case about this, have a look at the DIHCachedTest, check it out and let it run. You will need maven for this. I create an embedded HSQLDB that mimics your schema and run an embedded Solr that does have 2 data-import configurations.
- src/main/resources/solr/dih-cached/conf/data-config-fault.xml
- src/main/resources/solr/dih-cached/conf/data-config.xml
The first one is used for the test case DIHCachedTest#runImportWithFault. This will run into the same error you have posted in your question.
The second one is used for the test case DIHCachedTest#runImport. This will succeed.
If you compare both configurations, you will see that the only difference is the case of
cacheLookup="publication.CONFERENCE_ID" vs cacheLookup="publication.conference_id".
The issue SOLR-2483 DIH - an uppercase problem in query parameters describes this bug pattern.
回答4:
I dont think there is something wrong with your dataimporthandler configuration, I had rather check that your database types match your corresponding sold field type
Something like a database type resulting in a java.lang.Integer is passed to a solr field based on java.lang.String
OUPS, I might have got it wrong, is your uniqueKey field for SOLR still "id"? you've set in your schema this field as being an integer, which is a bad idea.
http://wiki.apache.org/solr/SchemaXml#The_Unique_Key_Field
"Note that if you have enabled the QueryElevationComponent in solrconfig.xml it requires the schema to have a uniqueKey of type StrField. It cannot be, for example, an int field."
回答5:
Try this Link http://wiki.apache.org/solr/DataImportHandler#CachedSqlEntityProcessor
Also you can refer https://issues.apache.org/jira/browse/SOLR-3850
and http://lucene.472066.n3.nabble.com/How-do-I-use-CachedSqlEntityProcessor-td4064919.html and http://lucene.472066.n3.nabble.com/Using-CachedSqlEntityProcessor-with-delta-imports-in-DIH-td4091620.html
回答6:
Solr CachedSqlEntityProcessor is depreciated somewhere around version 4.5. The new syntax uses cacheImpl, cacheKey & cacheLookup
I had to update the SQL in my data-config.xml
Old Syntax
processor="CachedSqlEntityProcessor"
where="fkId=parentTable.parentId"
New Syntax
cacheKey="fkId" cacheLookup="parenttable.parentId" cacheImpl="SortedMapBackedCache"
See full latest Solr wiki info, the old DataImportHanlder page is out of date https://cwiki.apache.org/confluence/display/solr/Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler#UploadingStructuredDataStoreDatawiththeDataImportHandler-EntityProcessors
来源：https://stackoverflow.com/questions/21549011/solr-dataimporthandler-cachedsqlentityprocessor-classcastexception