SharePoint Online, Viva , Teams, Power Platform, Azure, Identity

Delegate SharePoint Online CSOM (REST API) With .NET Core 3.1

Introduction

This article will cover, How to consume Sharepoint online CSOM (REST API) operation using .NET Core 3.1 framework into a console application.

The topic needs to cover here:

Create Azure AD Instance with Delegate Permission.
Create Console Application and Add Microsoft.SharePointOnline.CSOM Nuget Package.
Add Authentication Manager Class to generate the access token.
Get Access Token & access SharePoint Online List data

Step 1 - Create Azure AD Instance with Delegate Permission

Note

Users should have access to Azure Active Directory.

Login to https://portal.azure.com
Search and Select Azure Active Directory

Once the user selects Azure AD, the App registration page will appear.

Select App registrations
Select +New registration

Once the user selects new registration, add below details

Name "spocsom-api", user can add as per project namespace or naming convention guidelines.
Select "Account in this organization (single-tenant)
Click on the Register button

Grant access to SharePoint Online.

Select API Permission
Click to Add Permission
Search and Add SharePoint from Request API Permission

Delegate Permission, It required where user application needs to access data as the signed-in user.

Select Add a Permission
Selection Delegate Permission
Search and Select Sites
Choose "All Site.Read" -> This is not site specific but quite secure , It will just come into use to generate token, not going to grant access to the site to access the content.
Click to "Add Permission"

Grant Admin Consent

Grant permission by admin or user as part of the consent process.

Select "Grant admin consent for default directory and click ok to proceed.
All added api will show granted admin consent on behalf of the user. so it will not prompt for consent.

Allow Public Client Flows

The app is going to collect username and password into plain text, so it should allow to yes.

Select authentication and scroll down till the end
Under advance settings -> select Allow Public client flows -> Select Yes -> Click on Save to Proceed.

Step 2 - Create Console Application and Add Microsoft.SharePointOnline.CSOM Nuget Package

Login to Visual Studio 2019 and Create New Project
Select ConsoleApp (.Net Core) and Ok to proceed.

Install NuGet Package

Select solution and right-click on dependencies and NuGet Packages

Install the below packages:

Microsoft.SharepointOnline.CSOM
Newtonsoft.Json
System.IdentityModel.Token.Jwt

Step 3 - Add Authentication Manager Class to generate the access token

This class helps us to generate access tokens based on application URI, UserName, Password & Azurre AD App Client ID.

Create Auth Manager Class and Copy paste the below codebase:

public class AuthManager : IDisposable
   {
       private static readonly HttpClient httpClient = new HttpClient();
       private const string tokenEndpoint = "https://login.microsoftonline.com/common/oauth2/token";
       // Replace with Azure AD Client ID -Generated in above Steps
       private const string defaultAADAppId = "Azure Active Director Client ID";

       // Token cache handling
       private static readonly SemaphoreSlim semaphoreSlimTokens = new SemaphoreSlim(1);
       private AutoResetEvent tokenResetEvent = null;
       private readonly ConcurrentDictionary<string, string> tokenCache = new ConcurrentDictionary<string, string>();
       private bool disposedValue;

       internal class TokenWaitInfo
       {
           public RegisteredWaitHandle Handle = null;
       }

       public ClientContext GetContext(Uri web, string userPrincipalName, SecureString userPassword)
       {
           var context = new ClientContext(web);

           context.ExecutingWebRequest += (sender, e) =>
           {
               string accessToken = EnsureAccessTokenAsync(new Uri($"{web.Scheme}://{web.DnsSafeHost}"), userPrincipalName, new System.Net.NetworkCredential(string.Empty, userPassword).Password).GetAwaiter().GetResult();
               e.WebRequestExecutor.RequestHeaders["Authorization"] = "Bearer " + accessToken;
           };

           return context;
       }


       public async Task<string> EnsureAccessTokenAsync(Uri resourceUri, string userPrincipalName, string userPassword)
       {
           string accessTokenFromCache = TokenFromCache(resourceUri, tokenCache);
           if (accessTokenFromCache == null)
           {
               await semaphoreSlimTokens.WaitAsync().ConfigureAwait(false);
               try
               {
                   // No async methods are allowed in a lock section
                   string accessToken = await AcquireTokenAsync(resourceUri, userPrincipalName, userPassword).ConfigureAwait(false);
                   Console.WriteLine($"Successfully requested new access token resource {resourceUri.DnsSafeHost} for user {userPrincipalName}");
                   AddTokenToCache(resourceUri, tokenCache, accessToken);

                   // Register a thread to invalidate the access token once's it's expired
                   tokenResetEvent = new AutoResetEvent(false);
                   TokenWaitInfo wi = new TokenWaitInfo();
                   wi.Handle = ThreadPool.RegisterWaitForSingleObject(
                       tokenResetEvent,
                       async (state, timedOut) =>
                       {
                           if (!timedOut)
                           {
                               TokenWaitInfo internalWaitToken = (TokenWaitInfo)state;
                               if (internalWaitToken.Handle != null)
                               {
                                   internalWaitToken.Handle.Unregister(null);
                               }
                           }
                           else
                           {
                               try
                               {
                                   // Take a lock to ensure no other threads are updating the SharePoint Access token at this time
                                   await semaphoreSlimTokens.WaitAsync().ConfigureAwait(false);
                                   RemoveTokenFromCache(resourceUri, tokenCache);
                                   Console.WriteLine($"Cached token for resource {resourceUri.DnsSafeHost} and user {userPrincipalName} expired");
                               }
                               catch (Exception ex)
                               {
                                   Console.WriteLine($"Something went wrong during cache token invalidation: {ex.Message}");
                                   RemoveTokenFromCache(resourceUri, tokenCache);
                               }
                               finally
                               {
                                   semaphoreSlimTokens.Release();
                               }
                           }
                       },
                       wi,
                       (uint)CalculateThreadSleep(accessToken).TotalMilliseconds,
                       true
                   );

                   return accessToken;

               }
               finally
               {
                   semaphoreSlimTokens.Release();
               }
           }
           else
           {
               Console.WriteLine($"Returning token from cache for resource {resourceUri.DnsSafeHost} and user {userPrincipalName}");
               return accessTokenFromCache;
           }
       }

       public async Task<string> AcquireTokenAsync(Uri resourceUri, string username, string password)
       {
           string resource = $"{resourceUri.Scheme}://{resourceUri.DnsSafeHost}";

           var clientId = defaultAADAppId;
           var body = $"resource={resource}&client_id={clientId}&grant_type=password&username={HttpUtility.UrlEncode(username)}&password={HttpUtility.UrlEncode(password)}";
           using (var stringContent = new StringContent(body, Encoding.UTF8, "application/x-www-form-urlencoded"))
           {

               var result = await httpClient.PostAsync(tokenEndpoint, stringContent).ContinueWith((response) =>
               {
                   return response.Result.Content.ReadAsStringAsync().Result;
               }).ConfigureAwait(false);

               var tokenResult = JsonSerializer.Deserialize<JsonElement>(result);
               var token = tokenResult.GetProperty("access_token").GetString();
               return token;
           }
       }

       private static string TokenFromCache(Uri web, ConcurrentDictionary<string, string> tokenCache)
       {
           if (tokenCache.TryGetValue(web.DnsSafeHost, out string accessToken))
           {
               return accessToken;
           }

           return null;
       }

       private static void AddTokenToCache(Uri web, ConcurrentDictionary<string, string> tokenCache, string newAccessToken)
       {
           if (tokenCache.TryGetValue(web.DnsSafeHost, out string currentAccessToken))
           {
               tokenCache.TryUpdate(web.DnsSafeHost, newAccessToken, currentAccessToken);
           }
           else
           {
               tokenCache.TryAdd(web.DnsSafeHost, newAccessToken);
           }
       }

       private static void RemoveTokenFromCache(Uri web, ConcurrentDictionary<string, string> tokenCache)
       {
           tokenCache.TryRemove(web.DnsSafeHost, out string currentAccessToken);
       }

       private static TimeSpan CalculateThreadSleep(string accessToken)
       {
           var token = new System.IdentityModel.Tokens.Jwt.JwtSecurityToken(accessToken);
           var lease = GetAccessTokenLease(token.ValidTo);
           lease = TimeSpan.FromSeconds(lease.TotalSeconds - TimeSpan.FromMinutes(5).TotalSeconds > 0 ? lease.TotalSeconds - TimeSpan.FromMinutes(5).TotalSeconds : lease.TotalSeconds);
           return lease;
       }

       private static TimeSpan GetAccessTokenLease(DateTime expiresOn)
       {
           DateTime now = DateTime.UtcNow;
           DateTime expires = expiresOn.Kind == DateTimeKind.Utc ? expiresOn : TimeZoneInfo.ConvertTimeToUtc(expiresOn);
           TimeSpan lease = expires - now;
           return lease;
       }

       protected virtual void Dispose(bool disposing)
       {
           if (!disposedValue)
           {
               if (disposing)
               {
                   if (tokenResetEvent != null)
                   {
                       tokenResetEvent.Set();
                       tokenResetEvent.Dispose();
                   }
               }

               disposedValue = true;
           }
       }

       public void Dispose()
       {
           // Do not change this code. Put cleanup code in 'Dispose(bool disposing)' method
           Dispose(disposing: true);
           GC.SuppressFinalize(this);
       }

   }

Step 4 - Get Access Token & access SharePoint Online List data

Get List Data function get data from SharePoint online List and Print out all result based on return data.

static void Main(string[] args)
{
List<Data> getdata = GetListData();
foreach (Data data in getdata)
{
Console.WriteLine("Employee Name " + data.Title);
}
Console.ReadLine();
}

The codebase will invoke the SharePoint online api with help of an access token

public static List<Data> GetListData()
{
const string DataColumn = "ID,Title";
const string DataAPIAllData = "{0}/_api/lists/getbytitle('{1}')/items?$top=10&$select=" + DataColumn + "&$orderby=Modified desc";
try
{
var results = new List<Data>();
string sharepointSiteUrl = Convert.ToString("https://mittal1201.sharepoint.com/sites/CommSiteHub");
if (!string.IsNullOrEmpty(sharepointSiteUrl))
{
string listname = "Employee";
string api = string.Format(DataAPIAllData, sharepointSiteUrl, listname);
if (!string.IsNullOrEmpty(listname))
{
//Invoke REST Call
string response = TokenHelper.GetAPIResponse(api);
if (!String.IsNullOrEmpty(response))
{
JObject jobj = Utility.Deserialize(response);
JArray jarr = (JArray)jobj["d"]["results"];
//Write Response to Output
foreach (JObject j in jarr)
{
Data data = new Data();
data.Title = Convert.ToString(j["Title"]);
results.Add(data);
}
}
return results;
}
else
{
throw new Exception("Custom Message");
}
}
else
{
throw new Exception("Custom Message");
}
}
catch (Exception ex)
{
throw new Exception("Custom Message");
}
}
}

Created a data class with filed title here

public class Data
{
public string Title { get; set; }
}

Create TokenHelper class to get access token based on user credentials and URI. This function helps to get access from AuthManager Class with the provided information.

public static string GetAPIResponse(string url)
{
string response = String.Empty;
try
{
//Call to get AccessToken
string accessToken = GetSharePointAccessToken();
//Call to get the REST API response from Sharepoint
System.Net.HttpWebRequest endpointRequest = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(url);
endpointRequest.Method = "GET";
endpointRequest.Accept = "application/json;odata=verbose";
endpointRequest.Headers.Add("Authorization", "Bearer " + accessToken);
System.Net.WebResponse webResponse = endpointRequest.GetResponse();
Stream webStream = webResponse.GetResponseStream();
StreamReader responseReader = new StreamReader(webStream);
response = responseReader.ReadToEnd();
return response;
}
catch (Exception ex)
{
throw;
}
}
public static string GetSharePointAccessToken()
{
Uri site = new Uri("https://mittal1201.sharepoint.com/sites/CommSiteHub");
string user = "user email address";
string pwd = "user password";
string result;
using (var authenticationManager = new AuthManager())
{
string accessTokenSP = authenticationManager.AcquireTokenAsync(site, user, pwd).Result;
result = accessTokenSP;
}
return result;
}

Output Window

Execute this solution or press F5.

SharePoint List Screen Shot where data is going to read by a console app

Console Output

Finally, we got output using .NetCore 3.1 console app instead of .NetStandard framework.

I hope you enjoyed and learned something new in this article.

Publish Duplicate Client Side Page Changes To Original Page Within Same SPO Site Pages Library

Problem Statement

This article is a continuation of my previous article, Duplicate the Client Side Pages into same Modern SharePoint Online Library Folder where I explained how to duplicate the client side page with all existing controls.

In this article, I would like to explain this: If the user makes any changes to duplicate page, how to publish the duplicate client side pages to the original page into same site pages library within the same site collection.

As a developer, we have a couple of options to duplicate the page into another modern site collection library or different folder structure, but there is no way to copy the pages within the same modern site collection sites library within the same folder structure.

Prerequisite Steps

Let's create a page and add existing below control, or user can add any other available controls. This is just for demo purposes.

SharePoint Online PnP PowerShell Overview here
Browse the existing pages

Home Page looks like this,

Duplicate Page looks like this with changes,

Add new control
Modify the pages control layout

Approach Overview

Key steps to publish duplicate client side page with changes to original Modern Site Collection Site Pages into Same Library Folder.

Use PnP Online to connect to SharePoint Online Modern Site
Export PnP Client Side Pages Command export pages with PnP Provisioning Template
Store it locally.
Apply PnP Provisioning Template store it with different name.

try
   {

         $srcUrl = "https://mittal1201.sharepoint.com/sites/commsitehub"
         Connect-PnPOnline -Url $srcUrl

         $SourcePageName = "home_duplicate"
         $TargetPageName = "home"

         $tempFile = 'C:\CsharpCorner\'+ $SourcePageName +'.xml'
         Export-PnPClientSidePage -Force -Identity $SourcePageName -Out $tempFile

         $con = Get-Content $tempFile
         $sourcepage=$SourcePageName +".aspx"
         $targetpage=$TargetPageName +".aspx"

         $con | % { $_.Replace($sourcepage,$targetpage) } | Set-Content $tempFile
         Apply-PnPProvisioningTemplate -Path  $tempFile
         write-host -ForegroundColor Magenta "Page reverted with name of  " $targetpage


}
catch {
    Write-Host - ForegroundColor Red 'Error ', ':'
    $Error[0].ToString();

}

Save and run this script.

OutPut Steps

Applying template to client side pages

Export-PnPClientSidePage Cmdlets applying the PnP Provisioning template and export ".xml" file into shared locaiton. Cmdlets referece can be find here

Creation of client side pages

Apply-PnPProvisioningTemplate cmdlets will create a new page within the same library from download or exported ".xml" file with provided name i.e. pagename _ duplicate

Cmdlets referece can be find here

Final Duplicate Page Outcome

Browse the url; i.e., original page. New Client Side Page will be available with all configuration and controls.

I hope you have enjoyed and learned something new in this article. Thanks for reading and stay tuned for the next article.

Duplicate Client Side Pages Into Same Modern SharePoint Online SitePages Lib

Problem Statement

In this article, I would like to explain how to duplicate client side pages which have OOTB webparts and custom SPFx webparts into the same site pages library within the same site collection.

Prerequisites Steps

Let's create a page and add the exisitng below controls, or the user can add any other available controls. This is just for demo purposes.

SharePoint Online PnP PowerShell Overview here
Add and Configure Custom SPFx Control
Add and Configure Image Control
Add and Configure News Control

Approach Overview

We need to follow the below key steps to perform duplication of Modern Site Collection Site Pages into the Same Library Folder.

Use PnP Online to connect to SharePoint Online Modern Site
Export PnP Client Side Pages Command export pages with PnP Provisioning Template
Store it locally.
Apply PnP Provisioning Template store it with different name.

SharePoint Online PnP PowerShell Script

try
   {

        $srcUrl = "https://mittal1201.sharepoint.com/sites/commsitehub"
        Connect-PnPOnline -Url $srcUrl
        $pageName  = [System.Web.HttpUtility]::UrlDecode("Home")
        write-host $pageName
        $tempFile = 'C:\CsharpCorner\'+ $pageName +'.xml'
        Export-PnPClientSidePage -Force -Identity $pageName -Out $tempFile

        $con = Get-Content $tempFile
        $sourcepage=$pageName +".aspx"
        $targetpage=$pageName +"_Duplicate.aspx"

        $con | % { $_.Replace($sourcepage, $targetpage) } | Set-Content $tempFile
        Apply-PnPProvisioningTemplate -Path  $tempFile
        write-host -ForegroundColor Magenta "Page Created with name of  " $targetpage

}
catch {
    Write-Host - ForegroundColor Red 'Error ', ':'
    $Error[0].ToString();

Save and run this script.

OutPut Steps

Applying template to client side pages

Export-PnPClientSidePage Cmdlets applying the PnP Provisioning template and export ".xml" file into shared locaiton. Cmdlets referece can be found here

Creation of client side pages

Apply-PnPProvisioningTemplate cmdlets will create a new page within same library from download or exported ".xml" file with provided name i.e. pagename _ duplicate

Cmdlets referece can be found here

Final Duplicate Page outcome

Browse the url with suffix as _duplicate in the url. New Client Side Page will be available with all configuration and controls.

I hope you have enjoyed and learned something new in this article. Thanks for reading and stay tuned for the next article.

SPFX with DevOps using SharePoint ALM & Blob Storage

Implement SPFx (SharePoint Framework) deployment with DevOps using SharePoint ALM Commands & Blob Storage

Azure DevOps (Visual Studio Team Services / Team Foundation Server) consists of a set of tools and services that help developers implement DevOps, Continuous Integration, and Continuous Deployment processes for their development projects.

This article explains the steps involved in setting up your Azure DevOps environment with Continuous Integration and Continuous Deployment to automate your SharePoint Framework builds, unit tests, and deployment.

SharePoint ALM (Application Life Cycle Management) APIs provide simple APIs to manage deployment of your SharePoint Framework solutions and add-ins across your tenant or Site Collection level.

ALM APIs can be used to perform exactly the same operations that are available from a UI perspective

Continuous Integration

Continuous Integration (CI) helps developers integrate code into a shared repository by automatically build and packaging the solution each time new code changes are submitted.

Setting up Azure DevOps for Continuous Integration with a SharePoint Framework solution requires the following steps:

Office Azure & SharePoint ALM - Build Pipeline

Node 10.x
Gulp Clean
Gulp Build
Gulp Bundle --ship
Gulp Package-Solution --ship
Publish Build Artifacts (Task Version 1)

"Display Name" -> Publish Artifact: CDN
"Path to Publish" -> temp/deploy
"Artifact Name" -> CDN
"Artifact Publish Location" -> Azure Pipeline

Publish Build Artifact

"Display Name" -> Publish Artifact: App
"Path to Publish" -> sharepoint/solution
"Artifact Name" -> App
"Artifact Publish Location" -> Azure Pipeline

Continuous Deployment

Continuous Deployment (CD) takes validated code packages from build process and deploys them into a staging or production environment. Developers can track which deployments were successful or not and narrow down issues to specific package versions.

Setting up Azure DevOps for Continuous Deployments with a SharePoint Framework solution requires the following steps:

Office Azure & SharePoint ALM - Build Pipeline
Azure file copy (Task Version 2)

"Display name"-> AzureBlobFileCopy
"Source"-> $(System.DefaultWorkingDirectory)/_AzureDevOpsCI/CDN
"AzureConnectionType"-> Azure Resource Manager
"Azure Subscription"
"Destination Type"-> Azure Blob
"RM Storage Account"- >
"Container Name"->

SharePoint ALM: Catalog Scoped Actions

"Display name"->SharePoint ALM: Add Solution Package
"Connection to SharePoint" -> commsite01
"Action"-> Add Solution Package
"Path to SharePoint Solution Package"-> $(System.DefaultWorkingDirectory)/_AzureDevOpsCI/App/devops-01.sppkg
"
"PackageId output variable"-> AppPackageId

SharePoint ALM: Catalog Scoped Actions

"Display name"->SharePoint ALM: Deploy Solution Package
"Connection to SharePoint" -> commsite01
"Action"-> Deploy Solution Package
"Id of the package in the SharePoint App Catalog" -> $(AppPackageId)

End to End implementation SPFx deployment using Azure DevOps with SharePoint (Application Life Cycle Management) & Blob Storage for Static files.

SPFX with DevOps using Office 365 CLI

Implement SPFx (SharePoint Framework) deployment with Azure DevOps Pipeline using Office 365 CLI

Continuous Integration

Continuous Integration (CI) helps developers integrate code into a shared repository by automatically build and packaging the solution each time new code changes are submitted.

Setting up Azure DevOps for Continuous Integration with a SharePoint Framework solution requires the following steps:

Office 365 CLI Build Definition
Node 10.x
Gulp Clean
Gulp Build
Gulp Bundle --ship
Gulp Package-Solution --ship
Copy Files

"Display Name" -> Copy Files To: Copy Files to: $(Build.ArtifactStagingDirectory)/drop
"Contents" -> **/*.sppkg
"Target Folder" -> $(Build.ArtifactStagingDirectory)/drop

Publish Build Artifact

"Display Name" -> Publish Artifact: drop
"Publish To Path" -> $(Build.ArtifactStagingDirectory)
"Artifact Name" -> Drop
"Artifact Publish Location" -> Azure Pipelines

Continuous Deployment

Setting up Azure DevOps for Continuous Deployments with a SharePoint Framework solution requires the following steps:

Office 365 CLI Release Pipeline
Node 10.x
NPM

"Display Name" -> npm custom
"Command" -> custom
"command and arguments" -> install -g @pnp/office365-cli

Command Line

"Display Name" -> Connect to SPO
"Scripts" ->
o365 login https://$(tenant).sharepoint.com/$(catalogsite) --authType password --userName $(username) --password $(password)

Command Line

"Display Name" ->SP add solution
"Scripts" ->

o365 spo app add -p $(System.DefaultWorkingDirectory)/_SPFx-AutomateDeployment-CI/drop/drop/sharePoint/solution/clidemo-02.sppkg --overwrite --appCatalogUrl https://$(tenant).sharepoint.com/$(Devcatalogsite) --scope sitecollection

Command Line

"Display Name" -> Deploy Solution
"Scripts" ->

o365 spo app deploy --name clidemo-02.sppkg --appCatalogUrl https://$(tenant).sharepoint.com/$(Devcatalogsite) --scope sitecollection

End to End implementation SPFx deployment with Azure DevOps using Office 365 CLI