SPFx overclockers or how to significantly speed up the "gulp serve" command

A few months ago I wrote an article about SharePoint Framework build performance - SPFx overclockers or how to significantly improve your SharePoint Framework build performance. I've tried to reduce the amount of time the "gulp serve" command uses to re-build your code and to finally refresh a browser. I used different optimization technics for that purpose. The idea was to tweak the default SPFx build pipeline. However, those post only partially solves the problem. 

In this post, I will solve the problem from another way around (spoiler: I managed to make "serve" 10-15 times faster). 

The idea

How SharePoint Framework's "gulp serve" works? It gets your sources and outputs javascript bundles. But "gulp serve" becomes slow when your solution grows. How to fix that issue? Well, since it's slow, then don't use it! 

So this is the idea - don't use "gulp serve", use completely custom webpack based build to transform your sources into exactly the same javascript bundles which are produced by "gulp serve". 

What is the input for "gulp serve"? - TypeScript sources, styles. What is the output? - Javascript files. Can we get sources and produce exactly the same javascript? -Yes.

How it works

Some nerdy content goes below. If you don't want to read about internal implementation, go directly to "How can I use it?"

To make it work, we need custom webpack config and webpack dev server to serve webpack output. 

webpack config

The most essential part of the webpack config is below:

module: {
    rules: [
      {
        test: /\.tsx?$/,
        loader: 'ts-loader',
        options: {
          transpileOnly: true,
          compilerOptions: {
            declarationMap: false
          }
        },
        exclude: /node_modules/
      },
      {
        use: [{
          loader: "@microsoft/loader-cased-file",
          options: {
            name: "[name:lower]_[hash].[ext]"
          }
        }],
        test: /.(jpg|png|woff|eot|ttf|svg|gif|dds)((\\?|\\#).+)?$/
      },
      {
        use: [{
          loader: "html-loader"
        }],
        test: /\.html$/
      },
      {
        test: /\.css$/,
        use: [
          {
            loader: "@microsoft/loader-load-themed-styles",
            options: {
              async: true
            }
          },
          {
            loader: 'css-loader'
          }
        ]
      },
      {
        test: /\.scss$/,
        use: [
          {
            loader: "@microsoft/loader-load-themed-styles",
            options: {
              async: true
            }
          },
          'css-modules-typescript-loader',
          {
            loader: 'css-loader',
            options: {
              modules: true
            }
          }, // translates CSS into CommonJS
          "sass-loader" // compiles Sass to CSS, using Node Sass by default
        ]
      }
    ]
  },
  plugins: [
    new ForkTsCheckerWebpackPlugin({
      tslint: true
    }),
    new webpack.DefinePlugin({
      'process.env.NODE_ENV': JSON.stringify(process.env.NODE_ENV),
      'process.env.DEBUG': JSON.stringify(true),
      'DEBUG': JSON.stringify(true)
    })],

The key things here are loaders:

  • ts-loader transforms TypeScript to Javascript. It doesn't perform type checking (transpileOnly=true), thus works very fast. More on that below in plugins section
  • @microsoft/loader-cased-file - that's the same loader used by SharePoint Framework webpack task, it loads fonts, images
  • html-loader - loads HTML files. It's included into the default SharePoint Framework webpack task, so I added it as well
  • @microsoft/loader-load-themed-styles - that's also OOB loader, it transforms themable css into real css colors, thus very important
  • sass-loader - loads .scss files and transforms to .css
  • css-modules-typescript-loader - this loader transforms imports from *.module.scss files into TypeScript type definitions. 

Also, I use ForkTsCheckerWebpackPlugin. It performs type checks and listing in a separate asynchronous thread, thus it doesn't affect build performance. 

But it's not all. We also need "externals", "output" and "entry" values for our webpack config. Basically, these values are based on your SharePoint Framework solution configuration. And here is the trick - I have a pre-build task, which writes intermediate SPFx webpack config to the disk. Than when custom webpack build starts, it reads that config and sets correct "externals", "output" and "entry" values. 

webpack dev server

 We also need to serve our bundles. Webpack dev server is a perfect solution for that task:

devServer: {
    hot: false,
    contentBase: resolve(__dirname),
    publicPath: host + "/dist/",
    host: "localhost",
    port: 4321,
    disableHostCheck: true,
    historyApiFallback: true,
    open: true,
    openPage: host + "/temp/workbench.html",
    stats: {
      colors: true,
      chunks: false,
      "errors-only": true
    },
    proxy: { // url re-write for resources to be served directly from src folder
      "/lib/**/loc/*.js": {
        target: host,
        pathRewrite: { '^/lib': '/src' },
        secure: false
      }
    },
    headers: {
      'Access-Control-Allow-Origin': '*',
    },
    https: {
      cert: CertificateStore.instance.certificateData,
      key: CertificateStore.instance.keyData
    }
  },

Some key things here:

  • publicPath should be "https://localhost:4321/dist", since that's the location used by OOB "gulp serve"
  • I use proxy to re-map requests for localization resources from lib folder to src
  • for https configuration, I use OOB certificates (@microsoft/gulp-core-build-serve/lib/CertificateStore)

Customized gulpfile.js

In gulpfile.js we need two things - write intermediate webpack config to disk and ensure that workbench is created. It's possible to do that via below tasks:

const argv = build.rig.getYargs().argv;
const useCustomServe = argv['custom-serve'];
const fs = require("fs");
const workbenchApi = require("@microsoft/sp-webpart-workbench/lib/api");

if (useCustomServe) {
  
  const ensureWorkbenchSubtask = build.subTask('ensure-workbench-task', function (gulp, buildOptions, done) {
    this.log('Creating workbench.html file...');
    try {
      workbenchApi.default["/workbench"]();
    } catch (e) { }

    done();
  });

  build.rig.addPostBuildTask(build.task('ensure-workbench', ensureWorkbenchSubtask));

  build.configureWebpack.mergeConfig({
    additionalConfiguration: (generatedConfiguration) => {
      fs.writeFileSync("./temp/_webpack_config.json", JSON.stringify(generatedConfiguration, null, 2));
      return generatedConfiguration;
    }
  });

}

build.initialize(require('gulp'));

How can I use it?

To make it work, you need custom webpack.js file and patched gulpfile.js

To make your life easier, I've created a CLI tool, which does all the job - spfx-fast-serve. These are the required steps:

  1. npm install spfx-fast-serve -g
  2. Open a command line in a folder with your SharePoint Framework solution you want to speed up.
  3. Run spfx-fast-serve and follow instructions. In most cases, you shouldn't do anything specific and the cli "just works".
  4. Run npm install
  5. Run npm run serve and enjoy the incredible speed of serve command!

Which SharePoint Framework versions are supported?

The latest version of the tool supports the latest SPFx for SharePoint Online. 

SharePoint 2019 is also supported - but you need to run specific version via npx:

npx spfx-fast-serve@1.4.x

SharePoint 2016 is NOT supported.

Testing performance gain with different projects

I've tested the tool with various projects, including open-source ones. Check out the table below to see the difference in refresh time (the time needed to compile your project when you change a file and start refreshing a page in a browser) between gulp serve and spfx-fast-serve:

  gulp serve spfx-fast-serve
The default "Hello World"
React web part
3-5 sec 0.3-0.5 sec
PnP Modern Search solution 28-34 sec 2-4 sec
SP Starter Kit solution (v1) 40-50 sec 2-3 sec

NOTE: The actual time depends on the environment, hardware, but at least you can see the difference

The performance gain is significant for all types of projects. 

Final thoughts

spfx-fast-serve has below features/advantages over the default gulp serve:

  • works significantly faster
  • all compilation are done in a memory with webpack, no additional "copy", "prepare", "typescript", "whatever" tasks.
  • incremental TypeScript compilation when a file is being changed. It means only necessary files are compiled, not everything.
  • asynchronous type checking and linting.
  • supports local and hosted workbench
  • live reloading (for hosted workbench as well)
  • doesn't mess up your default SPFx build. If you have troubles, simply switch back to regular gulp serve

Please try it in your SharePoint Framework solution and tell me if it works for you. 

SharePoint Framework and React hooks. Should I care?

SharePoint Framework 1.9 introduced support for React 16.8+. While only a minor part of the version was changed (16.7 -> 16.8), it means a lot. It means that you can use the full power of React hooks. But should you? Obviously, the answer is yes, because React hooks introduce a lot of useful features, including:

  • reuse stateful logic across your many React components, which isn't possible with class-based components
  • get rid of high-order components, you can move some logic out of your React components into custom hooks. All that makes your code cleaner
  • all your React components now are in the same style (you don't mix class-based components with functional). Instead, you use only functional components, because they support state (with help of hook of course)

There are even more reasons going to hooks, instead of class-based components. Check out official documentation from React:

Should I use Hooks, classes, or a mix of both?

....we’d encourage you to start trying Hooks in new components you write. ....In the longer term, we expect Hooks to be the primary way people write React components.

Hooks are available starting from February 2019. A lot of libraries adopted their code to hooks. You are on the safe side if you're planning to use hooks in your code today. More...

SharePoint Framework tips & tricks: avoid css styles leaking from third party libraries

Sometimes when working on SharePoint Framework projects you have a need to use third-party libraries with their own css styles. You can include css styles using different technics - via import statement right in your code or using dynamic loading with SPComponentLoader. However, some css libraries have very common selectors, which affects Html in the "outside world".

For example, a library might include a css style for an element with class "some-class". This particular class might also be in SharePoint out-of-the-box styles. As a result, it breaks the UI:

It's called css leakage. More...

Show the progress of your PnP Provisioning process with SharePoint Application customizer and SignalR

The problem

You have a custom SharePoint Site Design, which executes (through MS Flow or Azure Logic App) PnP Provisioning process. You want to notify users that the site is not fully ready yet and it's still being updated (by background provisioning, which might take a long time). One option is to use two-way interactive communication between the SharePoint web site and the job using SignalR. That's something we're going to explore in this post in great detail. 

Check out below short video, which demonstrates the resulting UX we're building in this post:

The video was cut because the actual process takes 7-9 minutes on my tenant. 

Read further below to find out how to setup everything from scratch. 

All sources, as well as brief configuration steps, are available at the GitHub repository. More...

Recipe: Hub sites structure visualization without coding

The result

Ingredients

  1. Microsoft Flow - one piece
  2. Power BI report - one piece
  3. Modern SharePoint Power BI web part - one piece
  4. Mikael Svenson's post - Working with Hub Sites and the search API - one piece

Preparations

Well, actually we don't need any specific preparations. Just have a cup of tea or coffee if you wish :) 

The idea is that we query all hub sites and associated sites into a SharePoint list on schedule using MS Flow. Then we use Power BI with SharePoint as a data source to read hub sites data and visualize it on a report. More...

Exploring The Microsoft Graph Toolkit

At the end of September 2019, Microsoft released The Graph Toolkit library.

a collection of reusable, framework-agnostic web UI components that work automatically with Microsoft Graph

It was in preview for a while, now it's in GA, thus it's a good time to start exploring what is available in this library.  

First of all, that's a set of components, which abstract a lot of things about MS Graph, authentication, UI away from a developer. It provides a very seamless interface for building UI components. UI elements are built with web components technology - which means that they are framework agnostic. You can use it with any modern JavaScript framework.

Let's explore how to use this library with a React-based single page application and with SharePoint Framework. More...

SPFx overclockers or how to significantly improve your SharePoint Framework build performance

Please also check out this post - SPFx overclockers or how to significantly speed up the "gulp serve" command which uses different approach in performance tweaking and gives you extremely fast "serve" speed

Today's post will be about SharePoint Framework build performance. Especially about "serve" command, because it's the most frequently used command among developers. gulp serve is a kind of "watch" mode for your SharePoint Framework solution. As soon as you update a file, it will spin up the build process and will refresh your browser finally, so that you can see changes.

However, from here and there, I hear complaints about the poor performance of gulp serve command, especially if you have more than 10 web parts in a solution, or if your webparts are quite complicated (with lots of code and \ or additional heavy dependencies). Checkout Gulp webpack slow build and Long build times for SPFx projects with many components GitHub issues as well. I'm also not satisfied with the build performance in case of medium and of course big SharePoint Framework solutions. In a few recent weeks, I spent some time trying to go deeper and understanding all possible ways on how to improve performance for gulp serve command.

Read further and you will find a list of tricks, which reduce the amount of time to build a common SharePoint Framework solution. By build I mean serve or bundle (without --ship parameter) command, because they are very identical. The only difference is that serve is never-ending and has an additional step which refreshes your browser. In all other cases, they are the same, running tslint, typescript, sass, webpack, copy assets, etc. tasks. I will start with the easiest tricks, going to more complicated technics. I don't use any heavy hacks here. 

At the end of the post, you will find a detailed report on how any particular trick reduces build time on the example of SharePoint Starter Kit:

This is a solution designed for SharePoint Online which provides numerous web parts, extensions, and other components which you can use as an example and inspiration for your own customizations.

It contains 20+ webparts and quite slow when you use gulp serve command. Which makes it a good candidate for improvements. More...

Build SharePoint Framework solutions for on-premises SharePoint with ANY version of React, TypeScript or Office UI Fabric React

Any version?

-Yes, any.

Including the latest versions of React, TypeScript, etc. ?

-Yes.

The problem

SharePoint Framework is supported not only by SharePoint Online but by on-premises SharePoint as well (2019 and 2016 with Feature Pack). SharePoint Framework Yeoman generator has different options for different SharePoint versions and it generates different project templates depending on the environment selection.

On-premises SharePoint is always behind SharePoint Online in terms of features and codebase. And the same issue applies to SharePoint Framework. If you generate a "Hello world" SharePoint Framework web part for SharePoint 2019, you will see that it uses React 15.6, TypeScript 2.4 and Office UI Fabric React (OUIFR) 5.21. The most recent versions (as of today, August 2019) are React 16.8.6, TypeScript 3.5.3 and OUIFR 7.21.0.

Now you see the issue - you always have to work with an older version of packages. You miss a lot of potential features, bug fixes, and other things. Additionally, from a developer perspective, it's not exciting to work with outdated technologies or frameworks. The situation will be even worse in 2020 when we see React 17, TypeScript 4 and OUIFR 8. Will Microsoft update yeoman generator for on-premises to add support to the most recent version of packages? I don't think so. On-premises are not in the priority list today. 

For those who want to jump and explore the code right away - the full source code for this post is here at GitHub. More...

Using Lerna to manage SPFx projects with library components

Hydra is attacking porcupine? Well, actually not. Because Hydra is Lerna.js and porcupine is a SharePoint Framework solution with library components. Most likely you've heard about SharePoint Framework and library component, but not about Lerna. Lerna is

A tool for managing JavaScript projects with multiple packages.

Sounds simple, but probably still not very clear. First of all, it works only with JavaScript (and of course, TypeScript, for simplicity I use JavaScript term) projects. Some companies have JavaScript projects with lots if modules developed internally or publicly (it doesn't matter) in separate git repositories. These modules usually reference each other in the corresponding package.json files. Making changes across different modules is an extremely difficult and messy task. To solve these and other issues, some companies organize their projects in multi-packaged repositories (i.e. one git repository with many JavaScript packages):

Lerna is a tool that optimizes the workflow around managing multi-package repositories with git and npm.

Now it becomes a little bit clear what is Lerna. However, how does it correlate with SharePoint Framework and library components?

Imagine you have a solution with SharePoint Framework web parts and you want to add a new library component to share code across web parts. A new library component always creates a new JavaScript package (package.json). In the end, you will have two JavaScript packages in your repository - one for SharePoint Framework web parts and another one for the library component. So you have a multi-package repository and it's a good place to add Lerna to reduce the mess!

Actually, you're not limited in using Lerna with library components only. If you have a few separate SharePoint Framework solutions in one git repository, you can add Lerna to simplify package management. More...