Azure Synapse link for Dataverse - Introduction

Microsoft has recently announced that the Data Export Service (DES) will no longer be supported after 1 year (Nov-22). This may come as a surprise to many of the Clients who are using and depending on DES for their data extraction, integration and analytics. DES was one of the main tool used to extract and store data from Dataverse for analytical processing and integration purposes with other systems. 

There is a new option to get data from your Dataverse environments, store the data in a data warehouse and perform analytical and machine learning processing using that data. This is the Azure Synapse link for Dataverse.

So what is Azure Synapse? - In Microsoft speak - "Azure Synapse is an enterprise analytics service that accelerates time to insight across data warehouses and big data systems. Azure Synapse brings together the best of SQL technologies used in enterprise data warehousing, Spark technologies used for big data, Data Explorer for log and time series analytics, Pipelines for data integration and ETL/ELT, and deep integration with other Azure services such as Power BI, CosmosDB, and AzureML."

The main benefit of using Azure Synapse link for Dataverse is to get near real time insights on your Dataverse data. 

The below diagram shows the architecture of Azure Synapse. 

Azure Synapse Architecture

There are other benefits of using Azure Synapse link for Dataverse - 

1. You can utilize Synapse SQL, a distributed query system for T-SQL that enables data warehousing and data virtualization scenarios and extends T-SQL to address streaming and machine learning scenarios.

2. Enable Big Data and ML using Apache Spark for Azure Synapse - deeply and seamlessly integrates Apache Spark--the most popular open source big data engine used for data preparation, data engineering, ETL, and machine learning.

3. Use SQL and Spark together with data in your Azure Data Lake 

4. Built-in Data integration engine to ingest and create ETL pipelines at scale

5. Perform near real-time log and IOT stream analytics using Azure Data Explorer services

6. Use Azure Synapse Studio to build solutions, maintain and secure your data with a unified experience

Some of the challenges that you may need to consider -

1. Existing integration with external systems will need to be re-designed

2. Investments in non-Microsoft analytics and ML solutions may need to be evaluated against the above benefits.

3. Change management - The data engineering and analysis team will need to update their skills to be able to move to Azure Synapse.

4. There could be some cost benefits analysis that you will need to make to consider moving to Azure Synapse. Planning and budgeting for the move to Azure Synapse link for Dataverse is going to be critical for enterprise clients. 

To overcome some of the pain points and ease the transition from DES to Azure Synapse link for Dataverse, Microsoft has also provided a playbook. (link is here)  

Hope the above gets you started in planning to move from Data Export Services to Azure Synapse and do let me know about your experiences.

Thanks for reading.

@mihircrm

365WithoutCode 

Accelerate your Sales with Dynamics 365 - 2021 Wave-2

 Dynamics 365 has introduced many new and updated functionality in its 2021 Wave-2 release. This blog is part-1 of a series of blogs on the Dynamics 365 wave-2 release. This blog will focus on the updates on the new Sales accelerator (Sales - Customer Engagement app). 

The Sales accelerator focus is on empowering the Sales workforce to better collaborate (Teams), improve their productivity (intelligent worklist) and achieve better sales outcomes (smart sequences). The figure below shows the Sales Insights with conversational intelligence (in preview)

Sales insights with intelligence

Another exciting new functionality (which was lacking in Dynamics 365) is the ability to do a round robin assignment of leads. This will be only available in the Feb-2022 timeframe so I will write about it once the public preview is available.

1. Teams integration - The teams integration with Dynamics 365 was available earlier. There is new enhancements to the integration that allows the user to do the following (public preview starts in Dec 2021/Feb 2022 so will be able to get more details and post later). This is very useful if you are already using teams as a communication and collaboration tool within the organization.

    a.    Edit a D365 record from within a teams conversation

    b.    Ability to add a note or task to a D365 record from teams chat with the required security access

    c.    Share D365 records with other users in a teams chat as a interactable card

    d.    Create and join team meetings from D365 record. Access Sales records and add notes from meetings. 

    e.    Ability to @mention a user in D365 record timeline and the Sales bot will post a message to teams

    f.    Get notification of updates to D365 Sales records that you follow from within teams

Enhanced Teams integration

In addition to the above you can also add Conversation Intelligence to the teams call (in preview). This will provide the capability to analyze the call recordings and get insights on the customer interactions.

2. Worklist & Sequencing - Worklist allows the manager to setup a view of items that are assigned or created that the sales rep should be focusing on. The sales rep can than search, filter, sort the items on the worklist. The work list detail view shows the up next priority item that the seller needs to work upon based upon the sequencing as set-up by the manager.

Sales accelerator - Worklist 

3. Smart sequences - can be set-up to have the seller follow a guided particular sequence of to-do events or activities for working on a lead or opportunity. The below figure shows a new lead nurturing sequence steps. 

New lead nurturing sequence

This sequence can be configured and setup by sales managers and can automatically creates activities for the seller to complete as per the steps in the sequence.

The sequence needs to be assigned to the records (leads or opportunities) so that the sellers can follow the same while working on the sales records. The sequence assignment is shown below.

Assign sequence to lead records

I hope the above helps to accelerate your sales and help the sellers to better convert leads and close more opportunities with the wave-2 release of Dynamics 365 - Sales app. Please let me know if you would like to know more about the new Sales accelerator enhancements.

Thanks for reading.

@mihircrm

365WithoutCode

     

Where is your Data? - A need for Data governance platform

Digital transformation and app modernization programs results in new applications developed in the cloud or legacy apps moved to cloud. Along with the apps the data is also generated, moved, collected and stored in the cloud. The cloud data also needs to integrate and work with the data in onsite data repositories. As new data sources gets created it create new challenges for enterprises to track and manage data. 

Data storage and flows in hybrid cloud (image from Microsoft learn)

The consumers of data need to know where the data source is and how to connect to it. They also need to be able to trust the source of data. Data reliability is a concern when the data is used for AI and machine learning models and business insights generation.

The data producers needs to annotate, classify and create documentation for the data. Data management and maintenance is quite a challenging task for an enterprise with multiplying sources of data.

The data security admins needs to discover, identify, protect and restrict access to new and existing data sources. They need to make sure that proper process is followed when accessing sensitive classified data. Ensuring data classification, security and reliability is a concern for the admin with the increasing data generation and usage.

Finally there is the compliance aspects that needs to be met with Corporate data policies. This includes GDPR, HIPPA, PCI and other regulatory and industry standards. Data reporting is needed to ensure  compliance is met.  

This exponential growth and complexity in enterprise data in the cloud requires a cloud data governance platform.

The benefits of having a data governance platform can be -

1. An enterprise data catalog provide easy access to different source of data and information on those sources. This helps in better data discovery.

2.  Ability to make use of the data source and collaborate on the same with other users - for example asking for access 

3. Help in compliance - to make sure there is proper policy and procedure for enterprise data - manage the producing, consuming, storage and archiving of enterprise data, 

Microsoft has a cloud PaaS data governance platform called Azure Purview. Enterprise clients looking for Cloud data governance and compliance can benefit from using Azure Purview.  

I will write more about it once I have got some hands on experience testing Azure Purview with Power BI and Dynamics 365. How are you planning to manage and govern your data in cloud and onsite? Do you know where is your data? 

Thanks for reading.

@mihirCRM

365WithoutCode

Managing Environments in Power Platform - Risk Management Part-2

Environment in Power Platform is where you would store, share and manage your business data, Dynamics 365 and Portal apps, Power Automate flows, chatbots and connections. It is important to understand, control and manage your environments. This blogs provides some of the considerations in environment management for Power Platform / Dynamics 365 - Dataverse. The focus of this blog is on environments with Dynamics 365 apps & Dataverse database.

1. Environment Scope - Each Power Platform environment is created in the Azure AD tenant and bound to a geo location. Only users within the AD tenant can access the environment. An environment may only have a single Dataverse database.

Typically you will have multiple environments - Dev / Test / Staging / Production. Each environment will have its own Dataverse database. Any Power platform resource - app, chatbot, portal apps created within the environment will only connect to the environments Dataverse database. You can move, migrate solution and data from one environment to another. The data is stored in the environment's geo location data centers. This is important to know when storing customer data.

Environment and Tenant

2. Environment Types - The system will create a default environment. The default environment should not be used for building any Power Platform resource. Admin users can create multiple environments of type Sandbox. A sandbox environment can be converted to a type Production. You can have multiple Production environment within a Tenant. A production environment can also be converted to sandbox.

In addition there are other type of environment - Trial environment which are active for 30 days, developer environment for personal dev use and Microsoft Dataverse for Teams environment.

3. Environment Security Access -

To add a user to an environment the user needs to be added in the Azure AD Tenant, provisioned a license and also given the correct security role to access the data in the environment. There are 2 main roles Environment Admin and Environment Maker for Environments that do not have a Dataverse database.

Environments with Dataverse database have the following roles - 

Security Roles

The security roles can be assigned to a user, owner team and Azure AD group team. The Azure AD Group team can be Security or Office group team. The Azure AD groups can be used to manage licensed users apps and data access.  

4. Environment Backup and Restore - The system backs up all environments except Trial environment. The difference is only in how long the backups are stored. The Production environment backup with Dynamics 365 Apps are stored for 28 days. All other environments backups are stored for 7 days.

The backup is a continuous backup using Azure SQL Database. You can also do manual backups. You cannot download the backup. There are other options to download Dynamics 365 data such as using data migration. 

You can only restore a backup to a non production environment. To restore to a Production environment first convert the Production environment to Sandbox and than do the restore.

I hope the above helps you to understand and manage the Power Platform Environments with Dynamics 365 apps. Let me know if you have any questions. Thanks for reading. 

@mihircrm

365WithoutCode

Managing Risks with Microsoft Dataverse Power Platform and Dynamics 365 - Part-1 - Business Continuity & Disaster Recovery

Risks management is critical for every organization.  There was a time when the severity, definition and type of risks that a business may encounter were considered to be different. The risk was dependent on the size, nature of business and geographic location. Now that the core business apps and data are running in the cloud, the risks are the very much the same for every organization. 

Why risk management for the Cloud?

Cloud has leveled the playing field when it comes to risk management. Every organization big or small will face the same level of risk. The impact of the risk in $ value may be different. It is imperative for organizations to have a risk management plan to understand, analyze, mitigate and protect themselves from running their business in the cloud. This blog series is focused on understanding risk management with Microsoft Dataverse - Power Platform and Dynamics 365. (disclaimer - I am NOT a business risk management professional. This is my attempt to help you understand how risks can be managed when using Microsoft Dataverse)

#1 - Business Continuity and Disaster Recovery (BCDR)

When you purchase Microsoft Dynamics 365 or Power Platform licenses you will be provisioning 1 or more environments. Consider environment as a container for your business apps and data. You may have multiple development sandbox, testing, staging and production environments. Each environment has an instance of Microsoft Dataverse. Your business users are provisioned in the different environment. You may have your environments in 1 Geo region (lets say East Coast - US) which is nearest to your group of app users. You may even have users globally and have multiple environments in different Geo regions. How do you protect your environments from a unforeseen disaster in a geo region? How do you ensure that your mission critical business apps will be up and running asap and your data protected? 

Every cloud services provider has a well defined plan to protect their customers apps and data in the cloud. Microsoft has defined a Business Continuity and Disaster recovery plan for the Dataverse environments. 

Dynamics 365 Customer Engagement apps - for the production environments a replica or copy is made in a second geo region (Geo secondary replica), of the different storage services like Azure SQL and file storage and compute infrastructure. The copy is done at the time of the deployment and on an ongoing basis they are kept synchronized (with some lag in minutes) through continuous data replication. 

Dynamics 365 Customer Engagement - Geo Secondary Replica

Dynamics 365 Finance and SCM apps - The geo secondary replica is also done for this apps. 

Dynamics 365 - Finance and SCM apps

The replica environments are in a Azure region pair and have a dedicated low latency network. They also are within the data residency and compliance boundaries 

Azure region pairs

Failover types and service continuity - 

Unplanned failover - In case of an unplanned failover in an Azure geo region, Microsoft will notify customers and route traffic to the secondary geo replica region. There could be a possible data loss for about 15 minutes during the transition.

Planned failover - If there is a risk to the availability of the primary geo region (like an upcoming hurricane) Microsoft will notify the customers and transition to the secondary geo replica region. The users who are connected to Microsoft Dataverse apps may experience some disruption. There will be no data loss.

Failback - Once the primary geo region is back up and running Microsoft will notify customers and move to the primary region. Connected users may experience disruptions but there will be no data loss. 

Things to note - 

1. Dataverse for Team environments do not support the Geo Secondary replicas. So if you are using Dataverse for Teams environments please make sure to have a backup on a regular basis. 

2. The geo secondary replica is only available for production instances of Dataverse. All other instances will face service degradations till the primary regions is back up and running.

3. Dynamics 365 Finance and SCM apps will have limited functionality when working in the geo secondary replica region. The Financial Reporting and Power BI reporting services will not be available. Customers can open a support ticket with Microsoft to get the Financial Reporting setup running in the secondary region.

I hope this helps in understanding the business continuity and disaster recovery risk management for Microsoft Dynamics 365 - Dataverse platform. In my next blog I will write about Security and Data loss - Risks.

Thanks for reading.

@mihircrm

#365withoutcode

Measuring Customer Experience with Dynamics 365 Customer Voice

"What you cannot measure you cannot improve" - I think you will agree to this statement. If you want to improve your customer experience than first you need to know how to measure it. Once you measure it than you can track and take action from your insights to improve customer experience.

Customer Experience is about how the customer feels after having an interaction with your business. The interaction can be of 3 main types

1. Products - You want to know how the customer feels after purchasing and using your product. 

2. Services - If you are selling services, how the customer felt after the service was delivered. Like automotive maintenance service

3. Processes -  How was the customer online buying process experience or experience with the case resolution process.

An enterprise may want to know all 3 of the above and get valuable feedback from the Customer. If you have large number of products and thousands of customers globally you need to have a Enterprise Feedback management system like Dynamics 365 #CustomerVoice. 

I have written 2 blogs earlier about Microsoft Dynamics 365 - Customer Voice. My blog-1 gives an overview about Customer Voice and blog-2 is about considerations when deploying #CustomerVoice.

This blog is about how to measure your Enterprise Customer Experience using #CustomerVoice.

Measuring of Customer Experience is done using Satisfaction Metrics. The metrics are mapped to questions in the Survey. As the survey responses come in Customer Voice services sends the responses to Azure Cognitive services for analysis and also to the Customer Voice analytics engine. After the analysis is done the results are stores in Dataverse. 

Response analysis

There are 4 types of Satisfaction metrics that you can use. 

Satisfaction metrics

1. CSAT or Customer satisfaction is measured by responses to rating type questions in survey and is normalized to a range of 1-5. The below are example of CSAT survey questions.

CSAT question types

 

2. NPS or Net Promote Score is a metric used to measure customer loyalty. It is based on NPS type questions with scale 0-10 as shown below. 

NPS questions

Customer voice will group the respondents as follows 

    i.   Detractors = response from 0-6 rating

    ii.  Passive = responses with 7 or 8 rating

    iii.  Promotes = responses with 9 or 10 rating 

The NPS is a number calculated using the formula 

NPS = (% of Promoters) - (% of Detractors) 

NPS can range from -100 (100% detractors) to 100 (100% Promoters).

3. Sentiment - this identifies how the customer feels about a product, service or process. This is done using a text based question response. Sentiment will group the response based on the text as positive, negative or neutral. 

4. Custom Score - You may want to have a custom scoring of the question in a Survey. The custom score allows you to assign a point value to the answer options. Based on the response the score is calculated and assigned to the Survey.  This can help you to add more weightage to certain question answers and thus get an custom measurement of the respondents satisfaction level.

Custom Score

As you are planning and designing the questionnaire make sure that you understand how the customer experience can be measured so you have the right insights. In my next blog I will discuss about tracking and actionable insights.

I hope this helps you in improving your enterprise customer experience with #CustomerVoice. Thanks for reading.

@mihircrm

365WithoutCode

Teams and Power BI - Enable visual collaboration

 The integration of Power BI with Microsoft Teams has enabled enterprise wide collaboration on data and visual presentations. This brings new ways of generating team insights where remote teams can review the data and provide feedback to improve the visuals. 

Power BI and Teams - Visual collaboration 

There are multiple ways you can share the Reports and Visuals from Power BI service into a Teams Chat or Channel. This blog describes some of the options for using Power BI within Teams.

1. Power BI App in Teams - You can add the Power BI app to teams which will enable directly working in the Power BI Service from within Teams itself. 

Power BI app in Teams

2. You can also add a Power BI tab to a Teams Channel. This enables to collaborate on the Power BI report within the channel. 

Power BI - Add as a tab to Teams Channel

Note - When selecting to add a tab - you can only add a Power BI Report but not a dashboard to the channel.

3. Chat in Teams - You can chat in teams from a Power BI Report / Dashboard or even a visual. This inserts a link in teams chat for the shared content. 

 

Visual tile - Chat in Teams

4. Link preview - You can add a link to a Power BI - Report / Dashboard or App in Teams chat and a link preview is shown. The user can open the Power BI visualization from within Teams.

Link preview in Teams

Things to note -

1. In order for the Teams integration with Power BI - the users will need a Power BI Pro or Premium per user license or the Power BI Report is in a Power BI - Premium capacity. 

2. Users within the Team will need to be shared the reports and given access to the underlying dataset to enable them access to the content. One way to enable this is to share access to the Report workspace to the Microsoft 365 Teams group 

Microsoft 365 group access to Power BI workspace

3. Power BI dashboard cannot be added as a tab to Team channels. Also the Link preview does not work in Meeting Chat or Private Channels

4. Make sure that the Power BI Teams integration is enabled in the Admin Portal > Tenant settings. (by default it is enabled)

Teams integration is enabled

I hope that this helps you in integrating Power BI and collaborating within Microsoft Teams. Let me know how you have gained from the Team insights which are enabled with this new feature.

Thanks for reading

@mihircrm

365withoutCode