SPFx overclockers or how to significantly speed up the "gulp serve" command

A few months ago I wrote an article about SharePoint Framework build performance - SPFx overclockers or how to significantly improve your SharePoint Framework build performance. I've tried to reduce the amount of time the "gulp serve" command uses to re-build your code and to finally refresh a browser. I used different optimization technics for that purpose. The idea was to tweak the default SPFx build pipeline. However, those post only partially solves the problem. 

In this post, I will solve the problem from another way around (spoiler: I managed to make "serve" 10-15 times faster). 

The idea

How SharePoint Framework's "gulp serve" works? It gets your sources and outputs javascript bundles. But "gulp serve" becomes slow when your solution grows. How to fix that issue? Well, since it's slow, then don't use it! 

So this is the idea - don't use "gulp serve", use completely custom webpack based build to transform your sources into exactly the same javascript bundles which are produced by "gulp serve". 

What is the input for "gulp serve"? - TypeScript sources, styles. What is the output? - Javascript files. Can we get sources and produce exactly the same javascript? -Yes.

How it works

Some nerdy content goes below. If you don't want to read about internal implementation, go directly to "How can I use it?"

To make it work, we need custom webpack config and webpack dev server to serve webpack output. 

webpack config

The most essential part of the webpack config is below:

module: {
    rules: [
      {
        test: /\.tsx?$/,
        loader: 'ts-loader',
        options: {
          transpileOnly: true,
          compilerOptions: {
            declarationMap: false
          }
        },
        exclude: /node_modules/
      },
      {
        use: [{
          loader: "@microsoft/loader-cased-file",
          options: {
            name: "[name:lower]_[hash].[ext]"
          }
        }],
        test: /.(jpg|png|woff|eot|ttf|svg|gif|dds)((\\?|\\#).+)?$/
      },
      {
        use: [{
          loader: "html-loader"
        }],
        test: /\.html$/
      },
      {
        test: /\.css$/,
        use: [
          {
            loader: "@microsoft/loader-load-themed-styles",
            options: {
              async: true
            }
          },
          {
            loader: 'css-loader'
          }
        ]
      },
      {
        test: /\.scss$/,
        use: [
          {
            loader: "@microsoft/loader-load-themed-styles",
            options: {
              async: true
            }
          },
          'css-modules-typescript-loader',
          {
            loader: 'css-loader',
            options: {
              modules: true
            }
          }, // translates CSS into CommonJS
          "sass-loader" // compiles Sass to CSS, using Node Sass by default
        ]
      }
    ]
  },
  plugins: [
    new ForkTsCheckerWebpackPlugin({
      tslint: true
    }),
    new webpack.DefinePlugin({
      'process.env.NODE_ENV': JSON.stringify(process.env.NODE_ENV),
      'process.env.DEBUG': JSON.stringify(true),
      'DEBUG': JSON.stringify(true)
    })],

The key things here are loaders:

  • ts-loader transforms TypeScript to Javascript. It doesn't perform type checking (transpileOnly=true), thus works very fast. More on that below in plugins section
  • @microsoft/loader-cased-file - that's the same loader used by SharePoint Framework webpack task, it loads fonts, images
  • html-loader - loads HTML files. It's included into the default SharePoint Framework webpack task, so I added it as well
  • @microsoft/loader-load-themed-styles - that's also OOB loader, it transforms themable css into real css colors, thus very important
  • sass-loader - loads .scss files and transforms to .css
  • css-modules-typescript-loader - this loader transforms imports from *.module.scss files into TypeScript type definitions. 

Also, I use ForkTsCheckerWebpackPlugin. It performs type checks and listing in a separate asynchronous thread, thus it doesn't affect build performance. 

But it's not all. We also need "externals", "output" and "entry" values for our webpack config. Basically, these values are based on your SharePoint Framework solution configuration. And here is the trick - I have a pre-build task, which writes intermediate SPFx webpack config to the disk. Than when custom webpack build starts, it reads that config and sets correct "externals", "output" and "entry" values. 

webpack dev server

 We also need to serve our bundles. Webpack dev server is a perfect solution for that task:

devServer: {
    hot: false,
    contentBase: resolve(__dirname),
    publicPath: host + "/dist/",
    host: "localhost",
    port: 4321,
    disableHostCheck: true,
    historyApiFallback: true,
    open: true,
    openPage: host + "/temp/workbench.html",
    stats: {
      colors: true,
      chunks: false,
      "errors-only": true
    },
    proxy: { // url re-write for resources to be served directly from src folder
      "/lib/**/loc/*.js": {
        target: host,
        pathRewrite: { '^/lib': '/src' },
        secure: false
      }
    },
    headers: {
      'Access-Control-Allow-Origin': '*',
    },
    https: {
      cert: CertificateStore.instance.certificateData,
      key: CertificateStore.instance.keyData
    }
  },

Some key things here:

  • publicPath should be "https://localhost:4321/dist", since that's the location used by OOB "gulp serve"
  • I use proxy to re-map requests for localization resources from lib folder to src
  • for https configuration, I use OOB certificates (@microsoft/gulp-core-build-serve/lib/CertificateStore)

Customized gulpfile.js

In gulpfile.js we need two things - write intermediate webpack config to disk and ensure that workbench is created. It's possible to do that via below tasks:

const argv = build.rig.getYargs().argv;
const useCustomServe = argv['custom-serve'];
const fs = require("fs");
const workbenchApi = require("@microsoft/sp-webpart-workbench/lib/api");

if (useCustomServe) {
  
  const ensureWorkbenchSubtask = build.subTask('ensure-workbench-task', function (gulp, buildOptions, done) {
    this.log('Creating workbench.html file...');
    try {
      workbenchApi.default["/workbench"]();
    } catch (e) { }

    done();
  });

  build.rig.addPostBuildTask(build.task('ensure-workbench', ensureWorkbenchSubtask));

  build.configureWebpack.mergeConfig({
    additionalConfiguration: (generatedConfiguration) => {
      fs.writeFileSync("./temp/_webpack_config.json", JSON.stringify(generatedConfiguration, null, 2));
      return generatedConfiguration;
    }
  });

}

build.initialize(require('gulp'));

How can I use it?

To make it work, you need custom webpack.js file and patched gulpfile.js

To make your life easier, I've created a CLI tool, which does all the job - spfx-fast-serve. These are the required steps:

  1. npm install spfx-fast-serve -g
  2. Open a command line in a folder with your SharePoint Framework solution you want to speed up.
  3. Run spfx-fast-serve and follow instructions. In most cases, you shouldn't do anything specific and the cli "just works".
  4. Run npm install
  5. Run npm run serve and enjoy the incredible speed of serve command!

Which SharePoint Framework versions are supported?

The latest version of the tool supports the latest SPFx for SharePoint Online. 

SharePoint 2019 is also supported - but you need to run specific version via npx:

npx spfx-fast-serve@1.4.x

SharePoint 2016 is NOT supported.

Testing performance gain with different projects

I've tested the tool with various projects, including open-source ones. Check out the table below to see the difference in refresh time (the time needed to compile your project when you change a file and start refreshing a page in a browser) between gulp serve and spfx-fast-serve:

  gulp serve spfx-fast-serve
The default "Hello World"
React web part
3-5 sec 0.3-0.5 sec
PnP Modern Search solution 28-34 sec 2-4 sec
SP Starter Kit solution (v1) 40-50 sec 2-3 sec

NOTE: The actual time depends on the environment, hardware, but at least you can see the difference

The performance gain is significant for all types of projects. 

Final thoughts

spfx-fast-serve has below features/advantages over the default gulp serve:

  • works significantly faster
  • all compilation are done in a memory with webpack, no additional "copy", "prepare", "typescript", "whatever" tasks.
  • incremental TypeScript compilation when a file is being changed. It means only necessary files are compiled, not everything.
  • asynchronous type checking and linting.
  • supports local and hosted workbench
  • live reloading (for hosted workbench as well)
  • doesn't mess up your default SPFx build. If you have troubles, simply switch back to regular gulp serve

Please try it in your SharePoint Framework solution and tell me if it works for you. 

SharePoint Framework and React hooks. Should I care?

SharePoint Framework 1.9 introduced support for React 16.8+. While only a minor part of the version was changed (16.7 -> 16.8), it means a lot. It means that you can use the full power of React hooks. But should you? Obviously, the answer is yes, because React hooks introduce a lot of useful features, including:

  • reuse stateful logic across your many React components, which isn't possible with class-based components
  • get rid of high-order components, you can move some logic out of your React components into custom hooks. All that makes your code cleaner
  • all your React components now are in the same style (you don't mix class-based components with functional). Instead, you use only functional components, because they support state (with help of hook of course)

There are even more reasons going to hooks, instead of class-based components. Check out official documentation from React:

Should I use Hooks, classes, or a mix of both?

....we’d encourage you to start trying Hooks in new components you write. ....In the longer term, we expect Hooks to be the primary way people write React components.

Hooks are available starting from February 2019. A lot of libraries adopted their code to hooks. You are on the safe side if you're planning to use hooks in your code today. More...

SharePoint Framework tips & tricks: avoid css styles leaking from third party libraries

Sometimes when working on SharePoint Framework projects you have a need to use third-party libraries with their own css styles. You can include css styles using different technics - via import statement right in your code or using dynamic loading with SPComponentLoader. However, some css libraries have very common selectors, which affects Html in the "outside world".

For example, a library might include a css style for an element with class "some-class". This particular class might also be in SharePoint out-of-the-box styles. As a result, it breaks the UI:

It's called css leakage. More...

SharePoint development state in 2019: story based on sharepoint.stackexchange analysis with Power BI

The year 2019 is over and once again it's time to perform regular analysis of data at sharepoint.stackexchange. This is the fourth edition of such analysis. 

Tools used to collect and analyze data: 

  • Power BI with Power BI Desktop - super cool tools for data analysis. If you don't have experience with Power BI, it's worth trying to see what is possible. When I first tried it a few years ago I was sooo impressed with power yet simplicity in performing data analysis and building visualizations. It works very well for both simple and advanced scenarios. I believe that everybody will find these tools useful for any kind of data analysis. 
  • DaxStudio - extremely useful tool to test your DAX queries. 
  • Power BI Community - Power BI has a very strong community. I found a lot of answers at their forum, I even asked some questions and community helped with valid answers. That's not a "tool" but worth mentioning. I am grateful for all the answers.
  • Stack Exchange API

The source code used to gather initial data is available at GitHub. More...

Split your PnP Provisioning schema into multiple files for better maintenance

This post is for those, who have huge PnP Provisioning schema with hundreds or thousands of lines.

Or for those, who want to organize PnP provisioning schema into isolated logical components (folders & files) instead of having a solid single schema file.

When your PnP provisioning template grows, it becomes harder to search nodes, to add and update entities, to merge changes from other team members. Maybe you, reader, remember times of farm solutions (I've never thought that I will ever use this term in this blog in 2019, but anyway...) where all components were logically divided into different folders and files. We had different files for list schema, for fields, files, and for many other SharePoint artifacts. Having multiple files makes it easier to maintain such a solution (especially if you provision a lot of components). Additionally, your components are described in separate files and are referenced in different templates (instead of copy-pasting). In other words, you achieve schema reuse.

It's possible to have such logical separation in your PnP provisioning schema as well! Let's explore how to do it. More...

Show the progress of your PnP Provisioning process with SharePoint Application customizer and SignalR

The problem

You have a custom SharePoint Site Design, which executes (through MS Flow or Azure Logic App) PnP Provisioning process. You want to notify users that the site is not fully ready yet and it's still being updated (by background provisioning, which might take a long time). One option is to use two-way interactive communication between the SharePoint web site and the job using SignalR. That's something we're going to explore in this post in great detail. 

Check out below short video, which demonstrates the resulting UX we're building in this post:

The video was cut because the actual process takes 7-9 minutes on my tenant. 

Read further below to find out how to setup everything from scratch. 

All sources, as well as brief configuration steps, are available at the GitHub repository. More...

What’s new and what’s changed in SharePoint Online REST API in August-October 2019

I continue exploring REST API changes using my tool called SharePoint REST API Metadata Explorer. Here I post only some interesting findings, the full log is available for you on the SharePoint REST API Metadata Explorer web site under the "API Changelog" tab. 

Disclaimer

Please note, that all changes are gathered from Targeted tenant. Most likely these changes haven’t been officially introduced yet, use this post as spoilers to potential upcoming features. If you want to use APIs mentioned here in production, please check corresponding official documentation to make sure they are available.

AI, Machine Learning and Knowledge hub

The previous week was "Ignite" week. A lot of new things were introduced. Including innovations in the AI area. More...

Recipe: Hub sites structure visualization without coding

The result

Ingredients

  1. Microsoft Flow - one piece
  2. Power BI report - one piece
  3. Modern SharePoint Power BI web part - one piece
  4. Mikael Svenson's post - Working with Hub Sites and the search API - one piece

Preparations

Well, actually we don't need any specific preparations. Just have a cup of tea or coffee if you wish :) 

The idea is that we query all hub sites and associated sites into a SharePoint list on schedule using MS Flow. Then we use Power BI with SharePoint as a data source to read hub sites data and visualize it on a report. More...