etl

问题 I have a date 26.12.2019 (dd.mm.yyyy) in CSV which I'm trying to convert to 2019-12-26 using Derived Column in SSIS. I have used this expression but it does not seem to work. (TRIM([Period Start Date]) == "") ? NULL(DT_DATE) : (DT_DATE)(SUBSTRING([Period Start Date],7,4) + "-" + SUBSTRING([Period Start Date],4,2) + "-" + SUBSTRING([Period Start Date],1,2)) How do I rewrite this expression to produce the correct output with a value of data type? 回答1: When using DT_DATE data type, value is not

问题 I am using sqlalchemy and pandas to load dataframe to Oracle database. Since 'multi'method allows to upload in bulk, I choose that method. My pandas version is 1.0.1. However I got the error as following: The 'oracle' dialect with current database version settings does not support in-place multirow inserts. from sqlalchemy import create_engine oracle_connection_string = ( 'oracle+cx_oracle://{username}:{password}@' + cx_Oracle.makedsn('{hostname}', '{port}', service_name='{service_name}'))

问题 I have the following data: 12APR19 0644 and I want to combine/transform these in SSIS so they appear in my final destination as: 2019-04-12 06:44:00.00 I've tried using (DT_DBTIMESTAMP) [DateColumn] [TimeColumn] but this gives me a generic error. Anyone help please? 回答1: First transform your string to dd mmm yyyy hh:mm then use the type case dt_date: (DT_DATE)"12 APR 2019 06:44" returns: 4/12/2019 6:44:00 AM which you can further transform get the format you want. OTOH if your destination is

问题 Consider the following tasks: import luigi class YieldFailTaskInBatches(luigi.Task): def run(self): for i in range(5): yield [ FailTask(i, j) for j in range(2) ] class YieldAllFailTasksAtOnce(luigi.Task): def run(self): yield [ FailTask(i, j) for j in range(2) for i in range(5) ] class FailTask(luigi.Task): i = luigi.IntParameter() j = luigi.IntParameter() def run(self): print("i: %d, j: %d" % (self.i, self.j)) if self.j > 0: raise Exception("i: %d, j: %d" % (self.i, self.j)) The FailTask

问题 Consider the following tasks: import luigi class YieldFailTaskInBatches(luigi.Task): def run(self): for i in range(5): yield [ FailTask(i, j) for j in range(2) ] class YieldAllFailTasksAtOnce(luigi.Task): def run(self): yield [ FailTask(i, j) for j in range(2) for i in range(5) ] class FailTask(luigi.Task): i = luigi.IntParameter() j = luigi.IntParameter() def run(self): print("i: %d, j: %d" % (self.i, self.j)) if self.j > 0: raise Exception("i: %d, j: %d" % (self.i, self.j)) The FailTask

问题 I'm still new in SSIS, Now I can read the SQLStatement, FileName, and FileLocation based on this video But I'm facing the problems when export the flat file base on the File Location in SQL Table. You can check my sample package. Besides, the following screenshot shows the SQL table structure where information is stored: Different FileName, Different File Location and different SQL Command 回答1: Important Note Storing SQL Commands and file locations within databases and using them within a

问题 I want to build a data warehouse, and I want to use surrogate keys as primary keys for my fact tables. But the problem is that in my case fact tables should be updated. The first question is how do I find a corresponding auto-generated surrogate key for the natural key in the source system? I have seen some answers mentioning lookup tables which store correspondence between natural and surrogate keys, but I didn't understand how exactly they are implemented. Where this table should be stored:

问题 Context I am developing a simple SSIS package that contains a Data flow Task with: 1 OLE DB Source 2 Lookup Transformations 1 OLE DB Destination 1 OLE DB Destination for error output rows I am using the FastLoad option in both OLE DB Destinations and i have configured the error output of the first one to redirect rows to the second destination. Question From many online articles, i read that using Fastload option will cause the entire batch to fail an not only erroneous rows are redirected.

问题 Consider a situation where a task depends on another through a dynamic dependency: import luigi from luigi import Task, TaskParameter, IntParameter class TaskA(Task): parent = TaskParameter() arg = IntParameter(default=0) def requires(self): return self.parent() def run(self): print(f"task A arg = {self.arg}") class TaskB(Task): arg = IntParameter(default=0) def run(self): print(f"task B arg = {self.arg}") if __name__ == "__main__": luigi.run(["TaskA", "--parent" , "TaskB", "--arg", "1", "-

问题 I'm having the following issue: I have an SSIS package that connects to an oracle DB using Attunity Connection Manager. It does connect in my local pc (to a remote oracle db server) and extracts and all, seems to work. The problem comes when I load it to SQL server to be run as a Job. I have ran other jobs so the setup is proper, set to run with 32 bit runtime, loading as a file system type. but when I try to execute the job it gives the following error: Executed as user: username. Microsoft