SlideShare a Scribd company logo

Automatizacion de Procesos en Modelos Tabulares

•Download as PPTX, PDF•
•
Gaston Cruz
Gaston Cruz

Muestra de opciones para automatizar refrescos en Modelos Tabulares a traves de Azure Data Factory, Azure Logic Apps, Azure Functions y refresco de base de datos, tablas y particiones en Azure Analysis Services.

Related slideshows

RESTful services and OAUTH protocol in IoT
RESTful services and OAUTH protocol in IoTRESTful services and OAUTH protocol in IoT
RESTful services and OAUTH protocol in IoT
 
Understanding AJAX
Understanding AJAXUnderstanding AJAX
Understanding AJAX
 
SPSBE 2014 Content Enrichment in SharePoint Search
SPSBE 2014 Content Enrichment in SharePoint SearchSPSBE 2014 Content Enrichment in SharePoint Search
SPSBE 2014 Content Enrichment in SharePoint Search
 
1 of 16
Automatizacion de Procesos en Modelos Semanticos
Gaston Cruz - MVP Data Platform Data Analytics Architect Slalom, Seattle. /gastoncruz @gastonfcruz gastoncruz gastoncruz@outlook.com www.gastoncruz.com Automatizacion de Procesos en Modelos Semanticos
Contenido ● Arquitecturas ● Procesos ● Azure Logic Apps ● Azure Data Factory ● Resumen
Arquitecturas Azure Analysis Services Rest API 1 1 2 3 2 Azure Data Factory Web Activity Azure Logic Apps Http Action 3 App Registration Svc Principal The General purpose of this scenario is to orchestrate a processing of Azure Analysis Services objects from Azure Data Factory v2 4 4 Create a new App Registration (application services principal) in Azure Grant App Registration permissions to process Azure Analysis Services Model Create a Logic App that communicate with Azure Analysis Services REST API to process the model Create an Azure Data Factory pipeline with a Web Activity that will trigger the execution of the Logic App
Registro de Aplicaciones Go to Azure portal and search for App Registrations. On the overview click App Registrations followed by New Application registration
Registro de Aplicaciones In Settings of the new App Registration click Required Permissions and Select an API, type “Azure Analysis Services”. In Permissions select “Read and Write all Models”
Registro de Aplicaciones Back in the Settings of the App Registration, now click Keys. Enter a Key description, choose the duration. A key will be created, and we need to copy this key to paste in a following step
Registro de Aplicaciones Back in the Settings of the App Registration, now click Keys. Enter a Key description, choose the duration. A key will be created, and we need to copy this key to paste in a following step
Accesos y Permisos en AAS • Connect to your AAS server with SQL Server Management Studio. To process models using Rest API, the App Registered in before steps needs Server Administrator permissions. • Open AAS properties, Security and then Add. Add a manual entry using App ID and AAD ID (tenant). Use following syntax: app:<app guid>@<tenant guid>
Creando nuestra Logic App • Create a new Logic App from the Azure Portal Marketplace and click Edit • Pick the “When a HTTP request is received” as a trigger from the list of commonly used triggers • Click “+New step”, “Add an action” and choose the “HTTP – HTTP” action. • Now we have all the components needed to be triggered from ADF and to communicate with AAS Rest API.
Configurando nuestra Logic App Configure now the HTTP action (from Microsoft Official docs API: https://docs.microsoft.com/en-us/azure/analysis-services/analysis-services-async- refresh) Method: POST URI: https://<region>.as azure.windows.net/servers/<servername>/models/<modelname>refreshes Body: From the REST API documentation under POST/Refreshes with this sample we are refreshing the whole database (model) by a process type Full. It is possible to refresh only specific tables or partitions (see the image on how to refresh a table) { “Type” : “Full”, “CommitMode” : “transactional”, “MaxParallelism” : 2, “RetryCount” : 2, “Objects” : [] } Authentication: Active Directory Oauth Tenant: Use the AAD ID from the App Registration screen Audience: https://*.as azure.windows.net Client ID: Use the App Registration App ID from that screen Credential Type: Secret Secret: Use the App Registration Key from that screen
Realizando el Testing en nuestra Logic Apps • Run the Logic Apps from the Designer • Open the HTTP Action to see the REST API response code and msg • After the Logic Apps run and is ready to get executed from Azure Data Factory – Open the “When a HTTP request is received” trigger and take the HTTP POST URL
Creando nuestra instancia de Azure Data Factory • Create a new Azure Data Factory instance from Azure Portal • Define Name, Version, Subscription, Location and of course you can attach that instance to a GIT Repository (more later)
Diseño de Pipeline en Azure Data Factory • Go to Author & Monitor in the Azure Data Factory instance • In the visual pane, create a new pipeline and drag & drop a Web activity box • Configure the Web Activity: • URL: use the HTTP POST URL of the Logic App • Method: POST • Body (mandatory): { “message” : “test” } • Execute a Test Run and check
Definiendo Triggers en ADF • Go to Triggers (bottom left) • Define the Name, Description for the trigger • Setup a starting date • Recurrence (Every X minutes / hours / days / weeks / months) • Setup End Date
Gracias! Y nos vemos en una proxima charla! @gastonfcruz "Improve the Process"

More Related Content

Automatizacion de Procesos en Modelos Tabulares

  • 1. Automatizacion de Procesos en Modelos Semanticos
  • 2. Gaston Cruz - MVP Data Platform Data Analytics Architect Slalom, Seattle. /gastoncruz @gastonfcruz gastoncruz gastoncruz@outlook.com www.gastoncruz.com Automatizacion de Procesos en Modelos Semanticos
  • 3. Contenido ● Arquitecturas ● Procesos ● Azure Logic Apps ● Azure Data Factory ● Resumen
  • 4. Arquitecturas Azure Analysis Services Rest API 1 1 2 3 2 Azure Data Factory Web Activity Azure Logic Apps Http Action 3 App Registration Svc Principal The General purpose of this scenario is to orchestrate a processing of Azure Analysis Services objects from Azure Data Factory v2 4 4 Create a new App Registration (application services principal) in Azure Grant App Registration permissions to process Azure Analysis Services Model Create a Logic App that communicate with Azure Analysis Services REST API to process the model Create an Azure Data Factory pipeline with a Web Activity that will trigger the execution of the Logic App
  • 5. Registro de Aplicaciones Go to Azure portal and search for App Registrations. On the overview click App Registrations followed by New Application registration
  • 6. Registro de Aplicaciones In Settings of the new App Registration click Required Permissions and Select an API, type “Azure Analysis Services”. In Permissions select “Read and Write all Models”
  • 7. Registro de Aplicaciones Back in the Settings of the App Registration, now click Keys. Enter a Key description, choose the duration. A key will be created, and we need to copy this key to paste in a following step
  • 8. Registro de Aplicaciones Back in the Settings of the App Registration, now click Keys. Enter a Key description, choose the duration. A key will be created, and we need to copy this key to paste in a following step
  • 9. Accesos y Permisos en AAS • Connect to your AAS server with SQL Server Management Studio. To process models using Rest API, the App Registered in before steps needs Server Administrator permissions. • Open AAS properties, Security and then Add. Add a manual entry using App ID and AAD ID (tenant). Use following syntax: app:<app guid>@<tenant guid>
  • 10. Creando nuestra Logic App • Create a new Logic App from the Azure Portal Marketplace and click Edit • Pick the “When a HTTP request is received” as a trigger from the list of commonly used triggers • Click “+New step”, “Add an action” and choose the “HTTP – HTTP” action. • Now we have all the components needed to be triggered from ADF and to communicate with AAS Rest API.
  • 11. Configurando nuestra Logic App Configure now the HTTP action (from Microsoft Official docs API: https://docs.microsoft.com/en-us/azure/analysis-services/analysis-services-async- refresh) Method: POST URI: https://<region>.as azure.windows.net/servers/<servername>/models/<modelname>refreshes Body: From the REST API documentation under POST/Refreshes with this sample we are refreshing the whole database (model) by a process type Full. It is possible to refresh only specific tables or partitions (see the image on how to refresh a table) { “Type” : “Full”, “CommitMode” : “transactional”, “MaxParallelism” : 2, “RetryCount” : 2, “Objects” : [] } Authentication: Active Directory Oauth Tenant: Use the AAD ID from the App Registration screen Audience: https://*.as azure.windows.net Client ID: Use the App Registration App ID from that screen Credential Type: Secret Secret: Use the App Registration Key from that screen
  • 12. Realizando el Testing en nuestra Logic Apps • Run the Logic Apps from the Designer • Open the HTTP Action to see the REST API response code and msg • After the Logic Apps run and is ready to get executed from Azure Data Factory – Open the “When a HTTP request is received” trigger and take the HTTP POST URL
  • 13. Creando nuestra instancia de Azure Data Factory • Create a new Azure Data Factory instance from Azure Portal • Define Name, Version, Subscription, Location and of course you can attach that instance to a GIT Repository (more later)
  • 14. Diseño de Pipeline en Azure Data Factory • Go to Author & Monitor in the Azure Data Factory instance • In the visual pane, create a new pipeline and drag & drop a Web activity box • Configure the Web Activity: • URL: use the HTTP POST URL of the Logic App • Method: POST • Body (mandatory): { “message” : “test” } • Execute a Test Run and check
  • 15. Definiendo Triggers en ADF • Go to Triggers (bottom left) • Define the Name, Description for the trigger • Setup a starting date • Recurrence (Every X minutes / hours / days / weeks / months) • Setup End Date
  • 16. Gracias! Y nos vemos en una proxima charla! @gastonfcruz "Improve the Process"