Linear Merging: The Original Merge

In a Linear-Merge, you simply specify how much weight each model has on the resulting output...almost like merging models on Stable Diffusion.

A typical script will look like so:

models:
  - model: psmathur/orca_mini_v3_13b
    parameters:
      weight: 1.0
  - model: WizardLM/WizardLM-13B-V1.2
    parameters:
      weight: 0.3
  - model: garage-bAInd/Platypus2-13B
    parameters:
      weight: 0.5
merge_method: linear
dtype: float16

FrankenMoE Passthrough