azure-machine-learning-service

Azure ML PipelineData with DataTransferStep results in 0 bytes file

|▌冷眼眸甩不掉的悲伤 提交于 2021-01-27 22:15:02
问题 I am building an Azure ML pipeline with the azureml Python SDK. The pipeline calls a PythonScriptStep which stores data on the workspaceblobstore of the AML workspace. I would like to extend the pipeline to export the pipeline data to an Azure Data Lake (Gen 1). Connecting the output of the PythonScriptStep directly to Azure Data Lake (Gen 1) is not supported by Azure ML as far as I understand. Therefore, I added an extra DataTransferStep to the pipeline, which takes the output from the

How to write Azure machine learning batch scoring results to data lake?

非 Y 不嫁゛ 提交于 2021-01-27 22:02:02
问题 I'm trying to write the output of batch scoring into datalake: parallel_step_name = "batchscoring-" + datetime.now().strftime("%Y%m%d%H%M") output_dir = PipelineData(name="scores", datastore=def_ADL_store, output_mode="upload", output_path_on_compute="path in data lake") parallel_run_config = ParallelRunConfig( environment=curated_environment, entry_script="use_model.py", source_directory="./", output_action="append_row", mini_batch_size="20", error_threshold=1, compute_target=compute_target,

How to write Azure machine learning batch scoring results to data lake?

醉酒当歌 提交于 2021-01-27 18:32:11
问题 I'm trying to write the output of batch scoring into datalake: parallel_step_name = "batchscoring-" + datetime.now().strftime("%Y%m%d%H%M") output_dir = PipelineData(name="scores", datastore=def_ADL_store, output_mode="upload", output_path_on_compute="path in data lake") parallel_run_config = ParallelRunConfig( environment=curated_environment, entry_script="use_model.py", source_directory="./", output_action="append_row", mini_batch_size="20", error_threshold=1, compute_target=compute_target,

How to connect AMLS to ADLS Gen 2?

戏子无情 提交于 2020-12-11 02:33:07
问题 I would like to register a dataset from ADLS Gen2 in my Azure Machine Learning workspace ( azureml-core==1.12.0 ). Given that service principal information is not required in the Python SDK documentation for .register_azure_data_lake_gen2() , I successfully used the following code to register ADLS gen2 as a datastore: from azureml.core import Datastore adlsgen2_datastore_name = os.environ['adlsgen2_datastore_name'] account_name=os.environ['account_name'] # ADLS Gen2 account name file_system

export azure ml studio designer project as jupyter notebook?

瘦欲@ 提交于 2020-12-06 12:41:27
问题 I hope I am not missing something obvious here. I am using the new azure ml studio designer. I am able to use to create datasets, train models and use them just fine. azure ml studio allows creation of Jupyter notebooks (also) and use them to do machine learning. I am able to do that too. So, now, I am wondering, can I build my ML pipeline/experiment in ML studio designer, and once it is in good shape, export it as a python and jupyter notebook? then, use it in the same designer provided

AML - Web service TimeoutError

限于喜欢 提交于 2020-08-08 05:24:12
问题 We created a webservice endpoint and tested it with the following code, and also with POSTMAN. We deployed the service to an AKS in the same resource group and subscription as the AML resource. UPDATE: the attached AKS had a custom networking configuration and rejected external connections. import numpy import os, json, datetime, sys from operator import attrgetter from azureml.core import Workspace from azureml.core.model import Model from azureml.core.image import Image from azureml.core

Transfer from ADLS2 to Compute Target very slow Azure Machine Learning

岁酱吖の 提交于 2020-07-09 13:20:12
问题 During a training script executed on a compute target, we're trying to download a registered Dataset from an ADLS2 Datastore. The problem is that it takes hours to download ~1.5Gb (splitted into ~8500 files) to the compute target with the following method : from azureml.core import Datastore, Dataset, Run, Workspace # Retrieve the run context to get Workspace RUN = Run.get_context(allow_offline=True) # Retrieve the workspace ws = RUN.experiment.workspace # Creating the Dataset object based on