Azure integration with Dynamics 365 - Part 2 - Queues

This is Part 2 of the Azure integration with Dynamics 365. You can read the Part-1 blog about the Azure Service Bus.

In this blog we will set-up and configure Queue in Azure Service Bus to receive messages from Dynamics 365.
Queue is an entity within the Azure Service Bus which is used to store messages.

Benefits of using Queues in Azure -
1. Queues acts as an intermediary to communicate between 2 different systems.
2. It decouples the 2 systems - The sender can send messages and receiver can receive messages without them running at the same time.
3. Like a distribution center it receives products (messages) and sends to one or many systems. Note - each message can be received only by 1 receiving system.
4. Useful for batch and mobile applications

Consider a scenario - You wan to send data from Dynamics 365 to other line of business applications on the occurrence of an event in Dynamics 365. For example – when an Opportunity or an Order is created in Dynamics 365 Customer Engagement online, the details is to be sent to an on premise LOB application - like an ERP system.

In order to enable this the following is needed.

1. Azure subscription with Azure Service Bus and Queue
2. Service end point registration in Plugin Reg tool in Dynamics 365 to post to Azure
3. Set-up the LOB Application to listen to Azure. (Not covered in this blog)

The following diagram provides an overview of a Dynamics 365 to 3^rd Party application using Azure. (Ref-1)

The steps to set-up the Queue and Plugin is defined below.

1. Set-up an Azure subscription and create a Service Bus. The Opportunity service bus is created as shown below.

2. The Service Bus can have Queues or Topics. We will be using Queues to receive the Opportunity from Dynamics 365. In Azure, Go to Entity and select Queues and create a new Queue. The Opportunity queue is created as shown below.

3. Open the Queue and select Shared access policies. Create a new shared access policy and select manage checkbox. Open the shared access policy and copy the Connection String – Primary Key. We will be using it in the pluggin registration tool.

This is all for the Azure side. Now we are ready to configure the Dynamics 365 Service end-point.

4. Download the Dynamics 365 SDK. In the SDK, go to tools > plugin registration folder and run the plugin registration application.

5. Create new connection and enter the email address and password to connect to Dynamics 365 and login.

 6. Click register and select register new service end point. Here copy the connection string from Azure Service Bus portal

7. Paste the key copied in Step # 3 and click next. The Service end point registration is created. Click save.

Next we have to register a step to send Opportunities to Azure Service Bus.

8.    Right click on the Service End point and Select Register New Step.

9.    Select Create and enter primary entity as Opportunity as shown below. Set execution mode = Asynchronous. Select Authorization Type = SASKey. Message format can be as needed for the integration with the LOB application.

Once the step is registered it is ready for deployment.

To test the Dynamics 365 to Azure integration, I created 2 opportunities in Dynamics 365. 

The system jobs in D365, shows the system events which sent the 2 opportunities to Azure Service Bus as shown below.

And finally in the Azure Service Bus Queue, I can see the 2 Opportunity messages were successfully published.

The next step is to connect the LOB application to listen to the Azure Opportunity Queue. 

Hope the above provides details to do integration with Dynamics 365 and Azure. Thanks for reading.

Mihir Shah

Reference -

1. MSDN – Azure integration with Dynamics 365 - https://msdn.microsoft.com/en-us/library/gg334766.aspx#A%20Dynamics%20365-to-service%20bus%20scenari

Azure integration with Dynamics 365 - Part-1 - Azure Service Bus

In a connected world it is important for cloud based applications to be able to communicate and integrate with on premise and online business applications. Microsoft has OOTB and custom options that integrate Dynamics 365 with other business applications using the Azure platform. This blog is to discuss the Azure Service Bus options. The 2nd part of this blog will discuss the Dynamics 365 integration.

Why use Azure?

Azure Service Bus is a reliable, enterprise level message delivery service. The benefits of using Azure cloud messaging platform includes –

1.  Integrate between cloud to cloud or cloud to on premise business applications

2. Increase availability, security and scalability of systems

3. Cost effective and easy to manage

Azure Service Bus provides 3 communication mechanisms – Queues, Topics and Relays.

1. Queues – In a Queue, the sender sends a message to the Azure Service Bus Queue where it stays. When the Receiver connects to the Queue it gets the message. Queues are useful for batch and mobile applications.

Fig-1 - Queues

2. Topics – The sender sends a message to the Topic and it will stay there. There can be one or many receivers. The receivers can all receive the messages. The receiver can also filter the message they receive using their subscription.

Fig-2 - Topics with filtered Subscriptions

Relay – Relays provide 2 way / Direct communication link between Sender and Receiver. The Sender and Receive are both connected to the Relay using TCP/IP. This overcomes the Firewall and NAT problems faced with direct connections between 2 applications. Relay is created when the Receiver connects to Azure and a link is establish with the Sender. As soon as the connection drops the Relay is deleted.

Fig-3 - Relays - Direct link

Description	Queues	Topics	Relays
Communication	One directional	One directional	Bi-directional
Distribution	1 message to 1 recipient	1 message to many Recipients	Direct connection
Type	Asynchronous	Asynchronous	Synchronous
Filtering	No	Recipients can filter messages using subscriptions	No

In summary the right option needs to be selected for Azure Service Bus connection with business applications. In the next blog I will write about using Queues to publish data from Dynamics 365 to Azure Service Bus without writing any code.

Thanks for reading!!

Managing Solutions in Dynamics 365 - Part-2

As discussed in Part I of this blog, the solutions management is important for the ability to effectively maintain customization's between environments. Microsoft has provided OOTB capability to do solution versioning, patching and updating which is discussed in this blog.  

Solution segmentation / versioning.

The solution versioning can be in the following format – Major.Minor.Build.Revision

For example - AA.BB.CC.DD where,

AA = Major version #

BB = Minor version #

CC = Build #

DD = Revision #

1. Create a base solution – version 1.0.0.1

You can create a base solution in a source environment. We will include the entities and related assets, customization's…, that needs to be in the base solution. Let’s version this base solution as Release-1 version 1.0.0.1

And add the following assets –

Entity – Account – field – Account number

Entity – Contact – all assets

We can export the base solution as a managed solution and import the same into a Target environment.

2. Update base solution – Clone a Patch - version 1.0.0.2

Next lets us add a new custom entity called Hobby and add the same to the base solution. Now if we want to update the base solution we can create a patch of the base solution in Source environment.

Select the base solution and click on Clone a Patch button as shown below.

This will open another window where you can update the patch build and revision # as shown below.

Click Save.

Now there are 2 solutions, the base solution version 1.0.0.0 and a patch to the base solution version 1.0.0.1

In the source environment they will be as unmanaged solutions.

You can export the patch as a managed or unmanaged solution and import the same in the target environment where the base solution was imported. This blog will focus on deploying as managed solution.

Once a patch is created the base solution becomes a parent solution, it is locked in the source environment and no changes are allowed in the base solution. Also you cannot export the base (parent) solution from source environment.  If you delete all child patches of a parent solution, the parent solution becomes unlocked and it can be changed in the source environment.

You can create multiple patches for the base parent solution. More details on patches is given below (reference-1) -

·        A patch represents an incremental minor update to the parent solution.

·        A patch can add or update components and assets in the parent solution when installed on the target system, but it can’t delete any components or assets from the parent solution.

·        A patch can have only one parent solution, but a parent solution can have one or more patches.

·        A patch is created for unmanaged solution. You can’t create a patch for a managed solution.

·        When you export a patch to a target system, export it as a managed patch. Don’t use unmanaged patches in production environments.

·        The parent solution must be present in the target system to install a patch.

·        You can delete or update a patch.

·        If you delete a parent solution, all child patches are also deleted. The system gives you a warning message that you can’t undo the delete operation. The deletion is performed in a single transaction.

·        If one of the patches or the parent solution fails to delete, the entire transaction is rolled back.

3. Updating the solution – Cloning

Once all necessary patches are applied to a base solution you can update the solution using the Clone Solution button. Please note the following about updating solutions. (Reference-1)

·        When you clone a base solution, all child patches are rolled up into the base solution and it becomes a new version in the source environment. You can add, edit, or delete components and assets in the cloned solution.

·        A cloned solution represents a replacement of the base solution when it’s installed on the target system as a managed solution.

·        Typically, you use a cloned solution to ship a major update to the preceding solution.

Cloning a solution will combine the base solution and all the patches into an updated solution and also allow the solution version to be updated. As seen below we now have a Release 1 solution with version 1.1.0.0 in the source environment which is an upgrade of Release-1.

Now you can export the upgraded version as a managed solution and import in the Target environment.

The import process will detect this as an update and will offer the following import options.

1. Stage for upgrade – This option allows you to import the upgrade but not merge the solution with the base parent solution

2. Maintain customizations – This will allow you to maintain any unmanaged customizations in the target environment for the components in the managed solution

3. Overwrite customization's – This option will overwrite any unmanaged customization's in the target environment for the components in the managed solution

Once the update is imported the results window will show the button for applying the solution update.

You may want to not apply the solution update immediately. As this will combine the base solution and all the patches into the updated solution version 1.1.0.0.

The following figure shows the base solution with 2 patches and the upgrade solution in the target environment.

I would recommend to first do a test and make sure the customization's in the solution are all working as designed and then do the update of the solution. If there are any errors you can delete the version of the patch and the upgrade solution 1.1.0.0 and it will roll back the changes.

Once you are ok with the upgrade you can select apply solution upgrade button as shown below.

This will apply the upgrade and will update the solution to version 1.1.0.0

I hope the above helps in understanding the solution versioning, patching and updating process for managed solutions. It is important to take caution when deploying managed solution as even a small error can lead to lot of trouble. And if it is done correct, deploying managed solutions can eliminate many deployment issues and save considerable effort for the team.

.

Please do let me know your experience with managed solution and thanks for reading.

Reference-1 - MSDN - Create patches to simplify solution updates. - https://msdn.microsoft.com/en-us/library/mt593040.aspx

Managing Solutions in Dynamics 365 - Part-1

Solution deployment is always a major concern for Dynamics 365 projects. This blog is to discuss the OOTB options available for Solution management and focuses on Managed solution deployments in an online environment.

In a typical Dynamics 365 project you can have multiple deployments as the software customization code is moved using solutions from one environment to another.

Consider a solution as a bucket of components. Components is customization's done based on user story / requirements. So each solution is basically a group of user requirements delivered to environments for QA and UAT testing and finally deployed to Production.

Effective Solution management requires -

1. Ability to identify, track and control the solution versions deployed in each environment

2. Ability to differentiate between the software codes deployments between multiple environments

3. Ability to uninstall (rollback) to an earlier version if an error is detected in the current version of the solution (using managed solutions)

4. Ability to map and connect user story (requirements) to software customization's (components) 

Solution Types

There are 2 types of Solutions –

1. Managed Solutions

2. Unmanaged Solution

Most of the code deployments in the past versions of Dynamics 365 (CRM) was done using unmanaged solution. It was a challenge to work with Managed solution in a production environment. Also managed solution were mainly used by ISV’s for add-on package software distribution by Microsoft partners.

Some of the risks involved with using unmanaged solution in Production are –

1. It is difficult to control the code getting modified in a non-development environment.

2. It is not possible to roll back or undo the changes made with an unmanaged solution

3. Deleting an unmanaged solution does not delete the customization's in the solution

There are advantages to using Managed solutions -

1. A managed solution only includes solution components that have been customized.

1.1         This prevents unintentionally changing existing solution components on the system where the                                      solution is installed,

1.2         It also keeps the size of the managed solution smaller.

2. Managed solution helps in keeping control over any changes made in the non-development environments. 

You can defined what custom component can or cannot be customized in the managed environment. The managed properties have to be set in the unmanaged solution before exporting as managed solution.

3. Versioning and patching / updating is useful and easy to manage with OOTB functionality.

Risks involved with Managed solution – 

If a managed solution is deleted it deletes the following –

               1. All data stored in custom entities that are part of the managed solution

               2. All data stored in custom fields of system entities that are part of the managed solution

Hence it is never a good idea to uninstall or delete a managed solution as it will delete the data. You can update the managed solution thus maintaining the data.

Details about versioning, patching and updating managed solutions will be discussed in my next blog – Managing Solutions in Dynamics 365 Part-2.

Multiple Solutions –

Agile projects have a continuous development cycle. The solution will grow in size and complexity. It also requires multiple releases to production. It is a good practice to divide the solution into different multiple solutions so it is easy to manage and move from one environment to another. 

For example – You can divide your customizations and configuration components into 2 different solutions. It is also a good practice to have a separate solution for Security Roles to avoid user issues implications during deployments. So there can be multiple solutions like - Configurations / Customization's / Security Roles etc.

Resolving conflicts -

There can be a plethora of solutions OOTB from Microsoft (the last time I counted it has @14 managed solutions if you include all options), ISV solutions and custom development solutions which can be managed and / or unmanaged. 

It is important to understand how Dynamics 365 will resolve conflicts. The figure below shows how the different solutions interact with the System solutions resulting in the final application behavior. The system solution is the default OOTB solution. Managed solutions and Unmanaged customization's determine what the user will finally visualize and experience.

When there are multiple solutions Dynamics 365 will resolve conflicts using the two strategies – Merge and Top Wins. 

1. Merge – All UI components are re-calculated and merged from bottom to top. The unmanaged customization are last to be displayed.

2. Top Wins – All other non UI customization's the conflicts is resolved based on the solution components that is applied last.

Update to a Managed solution does not follow the above rule. 

If unmanaged customization's already exists for updates made to a managed solution, the user may not see the updated changes in the organization. The user will be given an option when updating the Managed solution whether to overwrite or preserve any existing customization's on top of the Managed solution. 

In addition the following is to be noted for updates to managed solutions

               1. Forms merging – Any form customization's may need to be further customized after the updates are published. 

                  

               2. Site Map – Site map customization's need to be verified and further changes may be needed as they are also                            getting merged.

In my experience it is better to have a pre-production staging environment to do a complete test before the solutions are deployed to production.

I hope the above helps in understanding and managing solutions. In my next part II of this blog I will write about the solution versioning and patching options.  

Let me know your past experience with solutions and if you are using Managed solutions for deployments.

Thanks for reading.

Mihir Shah

Monitoring Dynamics 365 using Organizational Insights

Once Dynamics 365 (CRM) is deployed to Production it becomes vital to monitor the health of the environment. Statistics that need to be monitored are - 

• Performance - Identify any issues and be able to troubleshoot quickly
• User adoption - Track # of users using the system 
• Usage & storage - Track what is used and capacity

The above would have required a custom dashboard in earlier version of Dynamics CRM. In the current version 8.2 of Dynamics 365 online, Microsoft has provided an app (there is an app for that!) to help in monitoring the instance and it is available on Microsoft AppSource.

 In this blog will walk through the installation process of Organization Insights and discuss the visualization available.

How to get Organization Insights – (only for version 8.2 and above)

1. The below link is for the Org. Insights app on App source.  Make sure to log in to the App source using your System admin Office 365 account and select the Get it now button. Agree to the terms and conditions.

1. https://appsource.microsoft.com/en-us/product/dynamics-365/mscrm.04931187-431c-415d-8777-f7f482ba8095?tab=Overview

2.  Select the correct organization and add the app to D365

3. The solution will be now seen in Installation Pending status in your Office 365 > Dynamics 365 Admin > Manage Solutions view. Confirm the solution shows as installed.

4. Once the solution status is installed in Office 365– confirm in Dynamics 365 that the managed solution is available.

5. In D365 go to > Settings > Admin > System Settings > Previews > Agree to T&C and enable Org Insights Preview. (This step may not be required in the future once Org Insights is out of preview) 

6. Now you should see the Organization Insights in Settings area 

Organizational Insights consist of dashboards (web resources) that has all the information needed to monitor, troubleshoot and track all in one place. It has the following dashboards - 

1. Home - This is the default dashboard that you can customize. You can add / remove / resize / re position the charts in this dashboard. There are currently 21 charts / views available to show on the home page. 

There are Date / Time filters also available to view different data over time. 

2. Active Usage - This contains information on the active users, api calls, CRUD operations in the specified time. 

    3. System Jobs - This dashboard provides details on the workflows & system jobs.

    4. Plug-ins dashboard - This shows the execution and performance of plugins.

    5. Storage dashboard - This shows the storage used by the tenant, instances and tables in D365.

    6. API Calls Statistics - This dashboard shows the API calls performance - useful for monitoring and troubleshooting of API calls

    

     As mentioned before Organizational Insights is a very useful tool for System Admins to monitor the performance, usage and storage of Dynamics 365 instances. For consultants it is one less thing to customize for D365.

    Hope you make use of Organizational Insights and do let me know about your experience with it.

    Keep reading !!

    Mihir Shah @MihirCRM

Managing Dynamics 365 (CRM) deployments online

Why we need to manage?

1. Multiple environments - 

Dynamics 365 (CRM) online enterprise projects have multiple instances deployed - Dev / QA / UAT.  Clients also want to be able to do continuous development and deliver releases to Production in an agile process. Agile also adds complexity, as we have multiple teams (dev / integration / business) who interact together as the development progresses through different sprints. Hence in a typical deployment you will have possibly many or all of the following instances.

1. Development
2. QA
3. UAT
4. Integration 
5. Migration
6. Performance
7. Pre-production
8. Production

2. Multiple user access - 

Each environment has its own team of users who may need access to that specific instance only. There may be few users who need access to multiple environments like Build Manager or System Admin. Also the Solution owner or Build manager maintains and deploy the correct customization code (solutions) for each instance. All of this requires proper planning and management of Dynamics 365 (online) deployments.

Here are some of the options to manage Dynamics 365 deployments -

1. Deployment process – A deployment process needs to be planned out for the Project. In an Agile continuous development the deployments are done in “Releases”. So there could be an initial Release-1 which goes through Dev to Production.  

A typical Release-1 deployment path from Dev to Prod is shown below -

Release-1 deployment path

 

Once the Production environment is ‘Live’ with Release-1 functionality, it is essential to have a second parallel path for the development and testing of Release-2 functionality. 

In addition a hotfix deployment path is also needed to fix any bugs found in Production environment. A Release-2 deployment can include the following instances.

Production and Release-2 development path

 

Any fixes done for Production are also implemented in Dev-2 environment. As the Release-2 is ready to be deployed the same path can be followed for deployments for the next Release cycle. Thus providing ongoing Production support and continuous development.

2. Control User Access - 

Security plays an important role in preventing unauthorized users from accessing environments. There are many ways that Dynamics 365 environments can be secured and managed.

1. Active Directory / Office 365 user access – Dynamics 365 online access is provided through Active Directory / Office 365. Once the user has access to Office 365,  they have to be provided access to the individual instances by assigning them a valid license and a security role. If an employees leaves the organization and their Active Directory access is removed, this also disables the user from all Dynamics 365 environment.
2. Security roles - This is the next step to grant user access to the Dynamics 365 instance. It is also probably the most widely used (not recommended) method of managing user access to different environments. A user with system admin security role in Dynamics 365 can assign another user a security role – This will grant access to the instance to the user.

If we only manage user access by adding or removing security roles it can be difficult to manage for large number of users. Users may get temporary access to environments and the access never gets removed. It is also a challenge because there is no way of knowing which user has access to what environments without logging in to each individual environment.

3. Security groups – This is one feature that should be used for managing user access to Dynamics 365 environments. In office 365 users can be added to Security groups. The security groups can than be assigned to the individual instances of Dynamics 365.

Users can be divided into the following groups.

1. Dev

2. QA

3. UAT

4. Integration / Migration

The security groups can now be given access to the corresponding environments. A user who needs access to multiple environments can be added to the corresponding security groups.

 

3. Solutions Management

 

Solutions are used to move customization's and configurations from one environment to another. The build manager will be responsible for the solution movements and tracking. Solutions can be managed or unmanaged and this depends on the project requirements. 

Dynamics 365 has tools to do Solution patching and versioning which can be utilized. Versioning is important to keep track of the customization's in each environment.

 

In addition to import and export a solution, there are 3 new buttons as highlighted above. They are

1. Clone a Patch

2. Clone Solution

3. Apply Solution Upgrade

I hope the above assists you in managing multiple Dynamics 365 deployments. Please leave your feedback on how do you manage Dynamics 365 deployments in your project. 

Keep reading.

Mihir

Dynamics 365 as a Collaboration Strategy tool

As we are starting the new year, there is the reflection of past experience and observations. One of which I want to highlight, is the increasing use of Dynamics 365 (CRM) as a collaboration strategy tool.

Why Collaborate - 

Since the dawn of internet, businesses worldwide are facing ever changing new challenges, like social media, app driven economy, consumer buying behavior, business models…

In order to manage the competitive forces and competition, firms are acquiring, merging or building strategic partnership. Some of the examples of strategic partnership or collaboration are -

• Automobile industry – Global partnership between auto manufacturers and parts suppliers to reduce cost of inventory and enhance quality
• International Air Travel industry - Strategic geographic alliances between airlines to attract the global corporate travelers and increase their market share
• Retail industry – Retailers adding new products and services and co-branding with firms to become the go to store / website / app for the customer.

The need for collaboration is growing and this is resulting in new business processes and systems designed to integrate business partners and customers to meet the shared goals.

How does Dynamics 365 (CRM) help in collaboration – 

Dynamics 365 has a set of built-in tools that can be effectively used for collaboration. The collaboration tools are discussed below.

1. Portals – Partner and Customer Portal in Dynamics 365 is the #1 tool for collaboration. Companies now have the capability of engaging their partners and customers with their core business teams and processes.
   
   1. Partners can be given secure access to the portal. Partners can provide input on sales opportunities, complete an activity or task and add / update documents as needed for the business process
   2. Customers can use the Portal to place orders, check history, request services and provide feedback which will be valuable to the organization
2. Mobility – You can provide mobile based access to partners with a mobile app. Using the mobile app the partners can log into Dynamics 365, access necessary information and update the records. For example a medical services firm can partner with independent nurses who visit patients, review their medical history and complete their annual health checkups at home using a mobile app.
3. Field Services – Dynamic 365 Field Services allows the scheduling of service providers and track maintenance of assets. The service providers can be partner firms who get their schedules from the Dynamics 365 and manage their daily service activities. This helps the services organization provide better service to the customer.
4. Security – One of the most important requirement for a collaboration system is Security. The security can be classified as –
   
   1. System level access – Restrict which employee of the partner gets access what part of the system
   2. Entity & record level access – Restrict what entity and records within the entity the partner can access - read and edit.
   3. Document access – Restrict access to documents and allow to add new documents
   4. Device access – What devices can be used to access and update data

Dynamics 365 with its granular security achieves all of the above and allows further control using business unit, teams, team queues, security roles, web roles (for portal), role based form…

5. Process flexibility – A collaborative system has to be flexible and adaptable to changing business processes. Dynamics 365 allows organizations to build upon an existing business process and add or change the process in the future as needed. Partners can be users of Dynamics 365 or can access the data through Portal or a mobile app. Within Dynamics 365 one can build custom business apps for Partners and provide a customized collaboration experience.

As you can see there are many new ways that Dynamics 365 can be used for Partner and Customer collaboration. I believe that more organizations will use Dynamic 365 as a Strategic Collaboration tool to engage partners and customers, improving the efficiency and productivity in their business process.

Please provide your experiences in using Dynamics 365 for collaboration.

Thanks for reading.

Mihir Shah