Migrate SharePoint on Premises document library’s or list to SharePoint Online using SPO API

SharePoint Point Online migration PowerShell is cmdlets is another ways of content migration without using the tool.

Below is the step by step SharePoint Online PowerShell cmdlets to migrate the content. It requires minimal CSOM (Client object model) call to avoid the threshold. It leverages temporary Azure Blob Storage container to hold the content and Azure Queue, which schedule parallel jobs.

Prerequisites to use the SPO Migration API

Here are the steps for using SPO Migration PowerShell to upload your on-premises data into SharePoint Online

Step1:- Export your document library content

This cmdlets export on-premises content to define folder structure.

Parameter

SiteUrl: - Source site url ( i.e. https://dev.northeurope.cloudapp.azure.com/sites/TeamSite)

ItemUrl: - relative path of document library or list (i.e. /sites/teamsite/contentlib )

Path: - folder structure path (C:\MigrationAPI\Contentlib)

Example

Export-SPWeb -Identity "<<SiteUrl>>" -ItemUrl "<<document library relative path>>" -Path "<<physical folder path>>" -NoFileCompression -IncludeVersions All

Step2:- Get your credentials into variables

$username = "manoj@sharepointtimes.onmicrosoft.com"

$cred = Get-Credential $username

Step3:- Set Source Package & File Path

At initial stage source file & package is same as export path of document library

$sourceFiles = "C:\MigrationAPI\Contentlib "

$sourcePackage=" C:\MigrationAPI\Contentlib "

Step4:- Set Output Package Path

$outputPackagePath = "C:\MigrationAPI\outputPackagePath"   

Step5:- Get target web & library

$targetWebUrl = "https://sharepointtimes.sharepoint.com/sites/teamsite"  

$targetLibrary = "contentlib"

Step6:- Convert Package for your target site

This creates a content package from a backup folder, the New-SPOMigrationPackage command reads the list of content targeted by the source path and will generate XML to perform the migration.

The following parameters are required:-

sourceFiles: points to the content you want to migrate

sourcePackage: points to content you want to migrate

targetWebUrl: point to your destination web

targetLibrary: point to the document library inside the web.

OutputPackagePath: points to your Temporary folder

Example:-

This example show how to convert a package to a target one by looking into source

ConvertTo-SPOMigrationTargetedPackage -SourceFilesPath $sourceFiles -SourcePackagePath $sourceFiles -OutputPackagePath $outputPackagePath -TargetWebUrl $targetWebUrl -TargetDocumentLibraryPath $targetLibrary -Credentials $cred

Step7:- Set Azure Storage Path & Variable

$fileContainerName = "onlinefile01”

$packageContainerName ="onlinepackage01"

$azureAccountName = "onlineazurestorage"

$azureAccountKey = “TQDKxHnHUDCG7uCjlfk646fzH5BV+mjjrocA9p6P0ENCJ2aHoY4+F6xOfdhnUslQ=="

$azureAzureQueueName = "migrationqueue01"

Step8:- Set SharePoint Online Migration Package source

This cmdlet help to create a defined file & package container into Azure blob storage as well as move the content from source file & output package path

The following parameters are required & optional:-

source files: points to the content you want to migrate

source package: points to your Temporary folder

FileContainerName:- temporary file container to hold the content (optional)

PackageContainerName:- temporary package container to hold the package (optional)

AccountName:- Azure blob storage account name

AccountKey:- Azure blob storage account access key

AzureQueueName:- Azure storage queue name (optional)

Example

$azurelocations=Set-SPOMigrationPackageAzureSource -SourceFilesPath $sourceFiles -SourcePackagePath $outputPackagePath -FileContainerName $fileContainerName -PackageContainerName $packageContainerName -AccountName $azureAccountName -AccountKey $azureAccountKey –Overwrite  -AzureQueueName $azureAzureQueueName

Step9:- List down the Azure Storage File container, Package Container & Queue URI

$azurelocations|fl

Step10:- Submit the Migration Job

This cmdlets help to generate the parallel queue to migrate the content from temporary storage to target web

There are three required parameter

TargetWebUrl:- target weburl where content need to migrate

MigrationPackageAzureLocations:- above set variable

Example

Submit-SPOMigrationJob -TargetWebUrl $targetWebUrl -MigrationPackageAzureLocations $azurelocations -Credentials $cred

Step11:- Monitoring the job status (Optional)

After the job is submitted, Only Azure Migration Queue interact to fetch and migrate the content to target SharePoint Online Site. This process is timer-job based.

Example

This cmdlet gives the running job status, If it doesn’t return any value – it means processing has been completed else it shows In Processing.

Get-SPOMigrationJobStatus -TargetWebUrl $targetWebUrl -Credentials $cred

Webinar Recording How SPO Migration API Works

Webinar Recording : 
https://youtu.be/BTGVTEWZpcM

Link to the PPT Slides :

https://www.slideshare.net/ManojMittal12/how-sharepoint-online-migration-api-works

Please keep checking my blog site for more webinars and useful articles.

Microsoft Flow Save update from Facebook to SharePoint List Part4

In this article, you will learn how to save update from Facebook a SharePoint List.

Part1:- http://manojmittalblogs.blogspot.in/2017/09/microsoft-flow-send-customized-email.html

Part2:- http://manojmittalblogs.blogspot.in/2017/09/microsoft-flow-save-my-email-attached.html

Part3:- http://manojmittalblogs.blogspot.in/2017/09/microsoft-flow-save-tweet-that-include.html

Step1:-

Navigate to Flow Page i.e. https://flow.microsoft.com/ and click My Flows and Create from template

Step2:-

Select Social Media from showing drop down & select the template named “Save update from Facebook to a SharePoint List”

Step3:-

Validate the account or use switch account if required and click continue.

Step4:- Fill template values

Select SharePoint Site and List name, where the post is going to store.

Give the name of flow (i.e. Part4)  and save it or create flow.

Step 5:- Login to Facebook account and post something on timeline

Step 6: Check the flow status, it shows succeed means it has completed successfully.

Step 7:- Navigate to SharePoint Site and List to check the tweet detail saved in a list or not.

Facebook post saved successfully in SharePoint List.

Microsoft Flow Save Tweet that include a special hashtag to SharePoint List Part3

In this article, you will learn how to save tweet that includes a  special hashtag to a SharePoint List.

Part1:- http://manojmittalblogs.blogspot.in/2017/09/microsoft-flow-send-customized-email.html

Part2:- http://manojmittalblogs.blogspot.in/2017/09/microsoft-flow-save-my-email-attached.html

Step1:-

Navigate to Flow Page i.e. https://flow.microsoft.com/ and click My Flows and Create from template

Step2:-

Select Social Media from showing drop down & select the template named “Save tweet that includes a special hashtag to a SharePoint List”

Step3:-

Validate the account or use switch account if required and click continue.

Step4:- Fill template values

1. Give the hashtag name i.e. (#DemoMicrosoftFlow)
2. Select SharePoint Site and List name, where tweet is going to store.
3. Select tweet and name which need to store in the title column.

Give the name of flow and save it or create flow.

Step 5:- Login  to twitter account and tweet using hashtag

Step 6: Check the flow status, it shows succeed means it has completed successfully.

Step 7:- Navigate to SharePoint Site and List to check the tweet detail saved in a list or not.

Twitter tweet saved successfully in SharePoint List.

Microsoft Flow Save my email attached to a SharePoint Document Library -Part2

In this article, you will learn how to save my email attached to a SharePoint Document Library.

Part1:- http://manojmittalblogs.blogspot.in/2017/09/microsoft-flow-send-customized-email.html

Step1:-

Navigate to Flow Page i.e. https://flow.microsoft.com/ and click My Flows and Click Create from template

Step2:-

Select Email from showing drop down & select the template named “Save my email attachment to a SharePoint document library”

Step3:- Validate the account or use switch account if required and click continue.

Step4:- Fill template values

1. Select the Outlook folder i.e. Inbox, Archive, Sent, Junk, Delete etc. to retrieve the attachments.
2. Select output from above step
3. Select the path of SharePoint Document Library to save the attachment.

Step 5:- Send an email with attachment to above-configured account. Login to O365 account email and check inbox email.

Step 6: Check the flow status, it shows succeed means it has completed successfully.

Step 7:- Navigate to SharePoint Site and Shared Document Library.

Email attachment saved in shared document library automatically.

Microsoft Flow: Send customized email when new item added to SharePoint list - Part1

In this article, you will learn how to send customized email, when new item added to SharePoint list.

Step1:- Navigate to O365 SharePoint Site e.g.

Step2:- Navigate to Site Content and create a custom list named “Experience-Flow”.

By default, New Experience is enabled and Flow is enabled for Modern List. If new experience enables, you can switch the using this article:-

http://manojmittalblogs.blogspot.in/2017/08/how-to-switch-list-experience-view-in.html

Modern List with Flow & Power App Options:-

Step3:- Click a create flow as highlighted above

Right-hand side option will appear. Select and click “Send a customized email when a new 

SharePoint list item added”.

Step 4:- You will be redirected to template page, where you can send the customized email to O365 account.

Step 5:- Verify the O365 account or you can switch to other accounts also and click continue.

Step6:- This template has pre-defined three section

1. Select the Site & List and associate the flow.
2. Select the user profile.
3. Select the email template using pre-defined fields

Step7:- Click edit and select the SharePoint site and list name.

Step8:- Click edit a send an email and modify the template as per requirement.

Email Template can be customized using advanced option:-

Step9:- Give the name and create flow

Step 10:- Create Item in the SharePoint List

Step 11:- Navigate to see your flow

Step 12:- Navigate to Outlook and you will get a mail on new item creation.

How to Switch List experience view in SharePoint Online

Introduction

In this article, you will learn, how to change the list or library experience in SharePoint Online.

1. New experience
2. Classic experience

New Experience set as default experience in SharePoint online, Site owner can easily switch with Classic experience.

How to Change the List Experience

1. Navigate to SharePoint Online site
2. Navigate to SharePoint Site List or Library for which you want to change the experience or view.

New Experience (Click List Settings)

In General Settings -> Select Advance Settings

In the List experience section, select the experience you want to use for the list

Classic Experience

Summary: Thus, in this article, you have seen how to change the list or library experience in SharePoint Online.

SharePoint Online Manual Migration

I worked with a couple of SharePoint Migration Project i.e.  On-Premises to On-Premises & On-Premises to Online using 

           Manual Approach ( Content DB Attach)

           ShareGate Tool 

           ShareGate Tool based PowerShell scripts.

           Manual Approach

In this article, I am going to articulate approach and scenario, where we used SharePoint Online Manual Migration approach.

Scenario: -

Migrate small size of content i.e. List & Library ~ 4-5 GB to SharePoint Online.

Why I chose manual approach?

Tool is really required if we have >25GB of content and another factor like:-

• changes in Application Architecture (Restructure the Content DB).
• Changes in Information Architecture or Schema.
• Version History needs to maintain.
• Metadata need to maintain i.e. Created By, Created, Modified and Modified
• Easy to get Pre & Post Migration reports & Verification.
• Incremental is just configuration away etc.
• Content Size is huge, Easy to schedule a day-night jobs.

Note: - Migration Tool use Content Migration API, which need cloud storage account. It also come up with extra cost along with Tool.

IF all above factors are not so important, we can easily use manual migration instead of tools.

How Manual Migration Done

Save Site as Template is not really possible using below factors:-

·         Publishing Site Template.

·         Publishing Feature activated in the site.

·         Sometime Managed Metadata Column and weightage of list column

2.       Save List as Template

This is one of easiest way to backup and restore the content. Even it support lookup column, 

If List backup and restoration configured in proper order i.e. First Parent then Child list.

Browse the List -> List Settings -> Save as Template. It will create and store the .STP file in the catalog library.

Another approach to create the STP file is PowerShell.

Both way, we can create the .STP file using Content or without Content.

Copy Data via Access:-

In some case, you have to use Access to move the content to Cloud. 

Select Open with Access

Specify the Path with filename or List Name, which create the access database (.accdb)

Two option available:-

• Link to Data on the SharePoint Site
• Export a copy of data

Once you download access database, Open the Database and select the list and Select SharePoint under Move Data Ribbon group. It prompts for destination list.

Once you complete above activity, you content will reside in the SharePoint Online List. It successfully support lookup column also.

Limitation: - If you have lots of view, It doesn’t maintain the sequence of column. Need to do again manually or using PowerShell script.

Copy Data via Excel:-

Use Export to excel, It export all list data into excel.

Below is the example of exports data.

Navigate to SharePoint Online List and click Quick Edit – It allow to copy paste the data from excel to list.

After copy the content, Its look like, At the one time, you can copy paste 500+ items also.

Click on Done and Job Finished.

Library Files:-

File Explorer View or Drag & Drop.  But it doesn’t allow more than 100 files at the same time.

Access allows you to copy list over to the cloud. But you can't use it for Libraries.
To copy files, use "Open with Explorer"

You still can copy these files by dragging and dropping directly in the document library:

When Manual Approach doesn’t suit (i.e. .STP file)

It doesn’t support two level below i.e. Moss2007 or SP2010 to O365. It is possible from SP2013 to O365.

How to set read-only

Once you transferred the data from source to destination, you must make sure that source data will not change. Manual Migration is tedious job and do the incremental is more tedious.

This can be achieved using Break permission inheritance and change to Read. Prioritize the list & library which are not updated frequently also help to avoid incremental for all.

Note:- Version History, All minor version should be published as major or discard.

Summary

If you have customization, you need to think, Can we convert to JS-Framework using REST or Search API, List form level customization using JS-Link.

If left behind customization, up to 4-5 GB content migration, we achieved in 8-12 hours jobs to avoid the incremental approach. To analyze and prepare the scripts and order of list took 2 weeks of time.