For Firefox on Windows, we have an MSI-based installer, but as its docs explain, it isn’t a “real” MSI; it doesn’t make any effort to use the MSI framework to run installer logic. Instead it’s a thin wrapper over the full installer. The MSI framework isn’t really supposed to let you do that, and neither is the WiX toolset that we use to generate the packages. So, how does it work? There’s a few aspects to it, and we’ll go through each one. Feel free to follow along in our WiX source file.

I’ll be assuming basic familiarity with MSI and WiX, so a quick look at its tutorial might be in order first.

Embed the executable in the MSI package

The first step is to get the executable installer itself bundled into the MSI package. Fortunately that’s easy; you only need one XML element and there’s nothing tricky or non-obvious about it:

<!-- This can go anywhere inside your Product element. -->
<Binary Id="WrappedExe" SourceFile="$(var.ExeSourcePath)" />

That’s it. Now our MSI contains a copy of the installer we want to run (inside its Binary table).

Run the executable

To get that installer to run, we’ll need to create a custom action and then add it to the install action sequence. The custom action we need to create looks something like this:

<!-- This can also go anywhere inside your Product element. -->
<CustomAction Id="RunInstaller" Return="check" Execute="deferred"
              HideTarget="no" Impersonate="no" BinaryKey="WrappedExe"
              ExeCommand="/S" />

Note that the BinaryKey attribute is set to the same string as the Id of the Binary element. And if you have any command-line parameters your installer needs (like a switch to make it run silently, as I’ve included here), add those in the ExeCommand attribute.

Now that we have an action defined, we need to get it to actually run by adding it to the sequence:

<InstallExecuteSequence>
  <Custom Action="RunInstaller" After="ProcessComponents" />
</InstallExecuteSequence>

Again here, the Action attribute’s value needs to match the Id of the CustomAction.

Add a few required boilerplate elements

MSI requires that we have at least one Cabinet, Component, and Feature, or it will refuse to load our file. So we’ll add in the WiX elements that create all those things:

<!-- The cabinet file doesn't actually exist, it's just a placeholder. -->
<Media Id="1" Cabinet="setup.cab" EmbedCab="yes" />

<Directory Id="TARGETDIR" Name="SourceDir">
  <Directory Id="TempFolder">
    <!-- Generate your own GUID for the component. -->
    <Component Id="EmptyComponent" Guid="c98a7cdd-1cba-4220-8d99-4ef977aeecff">
      <CreateFolder />
    </Component>
  </Directory>
</Directory>

<!-- Setting the feature to level 0 marks it hidden, so it can't be installed.
     That prevents getting this MSI registered as an installed product,
     because it has no features of its own to install. -->
<Feature Id="EmptyFeature" Level="0">
  <ComponentRef Id="EmptyComponent" />
</Feature>

As the comment says, having a single non-installable feature prevents any installed products, components, or features from being registered with MSI, which means installing our MSI won’t create an uninstall entry. Since the bundled installer is going to create its own uninstall entry, this trick prevents us from having two entries registered for the same product.

Handle MSI properties

The Firefox full installer can take a number of command-line arguments. We would like for the MSI to provide access to those, so anyone deploying the MSI is still able to customize their deployment, and we’d like for that access to be something typical to how other MSI installer work. So, we support customization in the form of MSI properties. For every possible command-line option, we implement a corresponding property and pass its value through to the installer command line.

To make that happen, first we’ll create a WiX Property for each thing we want to allow customizing. Here’s one of the several that are implemented for Firefox:

<Property Id="EXTRACT_DIR" Value="__DEFAULT__" />

Now, we need our custom action to implement these properties, that is, to insert them into the command line it invokes the installer with. For Firefox, we have an extra complication here: some of the command line arguments are mutually exclusive. That means we need some logic to determine which arguments we should include and which we should leave out, based on the ones that the administrator has configured with non-default values (that’s why we use a loud, conspicuous default string for the values that aren’t just a boolean flag).

What all that means is that we need a) more than one CustomAction, with the only difference between them being which set of mutually-exclusive arguments they include in their ExeCommand attribute, and b) for each CustomAction, one corresponding Custom element in the InstallExecuteSequence that includes the necessary logic to select it and only it to run when the right properties are set. These get pretty verbose in Firefox, so I’ll just show one pair of those elements as an example:

  <CustomAction Id="RunExtractOnly" Return="check" Execute="deferred"
                HideTarget="no" Impersonate="no" BinaryKey="WrappedExe"
                ExeCommand="/ExtractDir=[EXTRACT_DIR]" />

  <InstallExecuteSequence>
    <!-- Other Custom elements would be here. -->
    <Custom Action="RunExtractOnly" After="ProcessComponents">
      <!-- Only run this action when the EXTRACT_DIR property is set. -->
      <![CDATA[EXTRACT_DIR <> "__DEFAULT__"]]>
    </Custom>
  </InstallExecuteSequence>

And that’s everything, that’s pretty much how the Firefox MSI works.