flat-file

问题 Problem : I receive multiple sets of flat files on a weekly basis that need t be imported my Database. The flat files I receive are not in a conventional format to import, so they need to be run through a script and be parsed in to a more SQL friendly format. These flat files are usually in JSON, TXT, XML, LOG, ect. Current Solutions Current I have a windows forms application and another GUI to transform the files and to bulkimport to SQL tables. However, I'm finding it unreliable to ask

问题 Our client sends us a flat file as input, which we then take and convert to an XML file before sending to the destination system. The flat file consists of multiple lines, each line is delimited by LF or CRLF. How do I create a Flat File Schema so that BizTalk can interpret each line of data regardless of whether the line was delimited by LF (0x0A) or CRLF (0x0D 0x0A)? 回答1: Problem solved. Here is the solution in case anyone else is wondering: Since LF and CRLF both share the LF character, I

问题 I was just wondering what people's thoughts were on reading from Hive vs reading from a .csv file or a .txt file or an .ORC file, or a .parquet file. Assuming the underlying Hive table is an external table that has the same file format, would you rather read form a Hive table or from the underlying file itself, and why? Mike 回答1: tl;dr : I would read it straight from the parquet files I am using Spark 1.5.2 and Hive 1.2.1 For a 5Million row X 100 column table some timings I've recorded are

问题 First of all, I did spend quite some time on research, and I know there are many related questions, though I can't find the right answer on this question. I'm creating a SSIS package, which does the following: 1. Download and store CSV file locally, using HTTP connection. And 2. Read in CSV file and store on SQL Server. Due to the structure of my flat file, the flat file connection keeps giving me errors, both in SSIS as in the SQL Import Wizard. The structure of the file is: "name of file"

问题 Currently my batch has a static commit-interval (1000). We asked me to change it in order to commit once a data in my flat file get changed. So I should have a reader which reads lines from flat file, once it notice that this data has changed, it should process the read lines and writes in database. I tried with completionPolicy as follows: ReceptionCompletionPolicy.java : public class ReceptionCompletionPolicy extends SingleItemPeekableItemReader<ReceptionLineFieldHelper> implements

问题 I am new to hbase and I have a flat file(not delimited file) that I would like to load into a single hbase table. Here is a preview of a row in my file: 0107E07201512310015071C11100747012015123100 I know fo example that from position 1 to 7 it's an id and from position 7 to 15 it's a date.... The problem is how to build a schema that correspond to my file or if there is a way to convert it to a delimited file or read such file using jaql because I'm working with Infosphere BigInsights. Any

问题 I'm having a hard time trying to do a simple loading of data from a flat file to a database. The problem is that there are bad rows or at least rows that are not formatted as data in that text file. Sample.txt: Stackoverflow School at Philippines Record: 100101 Date: 6/20/2014 Name: Age: About: -------------------- --- -------------------------- Coolai 19 Bad Row Question Qwerty 17 Java Qwerty 19 C# *User1 21 Dynamic Data User4 27 Assembly Stackoverflow School at Nippon Record: 100102 Date: 6

问题 I'm a bit lost for I'm "green" in PHP. Please, may you teach me how to fix this: on 'POST' --> Replace a specified array key from a file: (WRONG:) <?php $newData = $_POST["sendData"]; if(isset($_POST['sendData'])){ $file = fopen('fileToOpen.php', 'a'); foreach($file as $key => $val) { $data[$key] = explode("|", $val); } for($k = 0; $k < sizeof($file); $k++) { unset($data[$k][3]); } $data[$k][3] = "$newData"; fwrite($file, $data[$k][3]); fclose ($file); } ?> That's wrong as it continues to

问题 I have an EBCDIC flat file to be processed from a mainframe into a C module. What can be a good process in converting the COMP and COMP-3 values into readable values? Do I have to convert the ebcdic characters to ascii then hex for COMP-3? What about for COMP? Thanks 回答1: Bill Woodger has given you some very good advice through his comments to your question, actually he answered the question and should have posted his comments as an answer. I would like to reiterate a few of his points and

问题 Presented alongside a polling facility on a webpage is the following input fields that enables the visitor (voter) to enter their details in a raffle for a hamper as a reward for their effort in taking part in the poll. The raffle entry form script is not attached to the poll script. All inputs in the raffle entry form script are validated and the info is sent to a flatfile. It is a short poll lasting only 7 days on different topics that might arise from time to time for a small town forum.