[Major Update] BitsandBytes Guidelines and Flux

[lllyasviel]

(Before we start, Forge now supports UI presets like this. Click this GIF to zoom in.)

---

(Again, before we start, to the best of my knowledge, I am the first one who made the BitsandBytes low bit acceleration actually works in a real software for image diffusion. You can cite this page if you are writing a paper/survey and want to have some nf4/fp4 experiments for image diffusion models.) -> Update Aug 12: It seems that @sayakpaul is the real first one -> they made so many contributions 😹

(BitsandBytes is a standard low-bit accelerator that are already used by most Large Language Models like LLama, Phi, etc)

@Misc{forge_bnb,
  author = {Forge},
  title  = {Forge GitHub Page about Flux, BitsandBytes, and Image Diffusion},
  year   = {2024},
}

---

Flux Checkpoints

The currently supported Flux checkpoints are

1. flux1-dev-bnb-nf4-v2.safetensors Full flux-dev checkpoint with main model in NF4. <- Recommended
2. flux1-dev-fp8.safetensors Full flux-dev checkpoint with main model in FP8.

Alternative: If you are looking for running original raw Flux, or GGUF, or any checkpoints that need loading separated modules like clip, t5, ae, etc, go to this post instead. <- If you want to run some 6GB flux unets, also use this link.

Some info about NF4:

(i) NF4 is significantly faster than FP8. To run Flux, NF4 is significantly faster than FP8 on 6GB/8GB/12GB devices and slightly faster for >16GB vram devices. For GPUs with 6GB/8GB VRAM, the speed-up is about 1.3x to 2.5x (pytorch 2.4, cuda 12.4) or about 1.3x to 4x (pytorch 2.1, cuda 12.1). I test 3070 ti laptop (8GB VRAM) just now, the FP8 is 8.3 seconds per iteration; NF4 is 2.15 seconds per iteration (in my case, 3.86x faster). This is because of less swap and partially because of NF4's native bnb.matmul_4bit rather than torch.nn.functional.linear: casts are avoided and computation is done with many low-bit cuda tricks. (Update 1: bnb's speed-up is less salient on pytorch 2.4, cuda 12.4. Newer pytorch may used improved fp8 cast.) (Update 2: the above number is not benchmark - I just tested very few devices. Some other devices may have different performances.) (Update 3: I just tested more devices now and the speed-up is somewhat random but I always see speed-ups - I will give more reliable numbers later!)

(ii) NF4 weights are about half size of FP8.

(iii) NF4 may outperform FP8 (e4m3fn/e5m2) in numerical precision, and it does outperform e4m3fn/e5m2 in many (in fact, most) cases/benchmarks.

(iv) NF4 is technically granted to outperform FP8 (e4m3fn/e5m2) in dynamic range in 100% cases.

This is because FP8 just converts each tensor to FP8, while NF4 is a sophisticated method to convert each tensor to a combination of multiple tensors with float32, float16, uint8, int4 formats to achieve maximized approximation.

(Do not confuse FP8 with bnb-int8! In large language models, when people say "8-bits better than 4 bits", they are (mostly) talking about bnb’s 8-bit implementation, which is a more sophisticated method that also involve storing chunked float32 min/max norms. The fp8 here refers to the naked e4m3fn/e5m2 without extra norms. ) <- You can say that bnb-8bit is more precise than nf4. But e4m3fn/e5m2 may not.

For example, this is the Flux FP8 state dict (you can see that each weight only has one FP8 tensor):

This is the NF4 state dict, you can see that each weight is stored in 6 different tensors, and these tensors are in different precisions including float32, uint8, etc:

So, do not be surprised if you find out that NF4 is actually more precise than FP8 despite its smaller size. And, do not argue too much if you still find FP8 more precise in some other cases ...

(The short version of the real method is that NF4 sort values in a tensor and split in into chunks and compute the abs norm of each chunk. Such norm is stored in higher precision. The “N” in NF4 means “Nested” (Update: "NormalFlow"), which means this compression happens two times. The first time it processes tensor weights, and the second time it processes extracted norms.) <- correct me if I am wrong

(The long version of the method is here)

Using Flux

If your device supports CUDA newer than 11.7 then you can use NF4. (Most RTX 3XXX/4XXX GPUs supports this.) Congratulations. Enjoy the speed. In this case, you only need to download the flux1-dev-bnb-nf4.safetensors.

If your device is GPU with GTX 10XX/20XX then your device may not support NF4, please download the flux1-dev-fp8.safetensors.

In the UI, Forge gives you an option to force the loading weight type:

However, in most cases, you can just set it to Auto and then it will use the default precision in your downloaded checkpoint.

Using this option, you can even try SDXL in nf4 and see what will happen - in my case SDXL now really works like SD1.5 fast and images are spilling out!

Important: Do Not Load FP8 checkpoint flux1-dev-fp8.safetensors with NF4 Option! If you do this, you will still get images eventually with no errors. However, you will waste 30 seconds to de-quantize the weight into fp16 and then waste 60 seconds quantize it again into NF4. And finally, your results are worse than flux1-dev-bnb-nf4.safetensors because the model is quantized twice, and quality is lost.

Getting Theoretical Upper Bound of Inference Speed

Forge's default preset should already be very fast (the default preset is already among the fastest ones, with about +/- 5% speed variations, now on my several devices even using FP8, compared to all SD-related software without special packages like TensorRT, if timed using my phone outside screen).

But you can make it even faster and really get the theoretical upper bound of inference speed for every different GPU by tuning the UI.

To begin with, this is your Flux model, and lets use the FP8 model as an example, where the base diffusion model is about 11 or 12GB.

Then lets say this is your system, you have 8GB VRAM, 32GB CPU memory, and a block called "shared GPU memory" with 16GB

Then you want to move the model to GPU. But the model is 12GB, bigger than your GPU memory 8GB. Then how?

The anwser is to split the model to two parts. One part to GPU, one other to a "swap" location.

If you select CPU as swap location, your model will load to CPU memory and GPU:

If you select "Shared" as swap location, your model will load to GPU and shared memory:

On newer devices, the "Shared" offload/swap is about 15% faster than CPU swap. However, some other devices reports crash when using "Shared" swap.

Then you can select the maximum memory to load model to GPU.

Below is an example using Flux-dev in diffusion:

Another example:

Larger GPU Weights means you get faster speed. However, if the value is too large, you will fallback to some GPU problems and the speed will decrease to like 10x slower.

Smaller GPU Weights means you get slower speed. However, you will be able to diffuse larger images, because now you have more free VRAM.

Okay then the last thing, Swap Method:

Queue: you will load a layer to GPU, and then compute, and then load another layer, and then compute, ... just like it is a queue.

ASYNC: You will have two workers. One worker always compute layers. One worker always load layers to GPU. They work at the same time.

Some experiments report that the ASYNC method is 30% faster than Queue. However, there is a drawback that one worker may mistakenly moved too many layers to GPU, making the other worker cannot have enough GPU memory to compute. In that case, the speed will be suddenly 10x slower.

Based on the above info, you can now tune the best config for your device:

Once you are done, you are on the theoretical upper bound of inference speed for your device (without using methods like TensorRT that degrade compability to like some control models).

Finally, you may need to turn on system swap if it crash:

Distilled CFG Guidance

Flux-dev is a distilled model. It is recommended to set CFG=1 and then do not use negative prompts. Using “Distilled CFG Guidance” instead. The default value is 3.5.

Note that if CFG=1, the UI of negative prompt will be greyed out.

Sanity Check

Finally, use this as a sanity check! (thanks for the prompt)

Astronaut in a jungle, cold color palette, muted colors, very detailed, sharp focus

Steps: 20, Sampler: Euler, Schedule type: Simple, CFG scale: 1, Distilled CFG Scale: 3.5, Seed: 12345, Size: 896x1152, Model: flux1-dev-bnb-nf4-v2

Make sure that you get similar image:

---

Update 1:

Hi everyone, after tested with more devices, the speed-ups of bnb are more random than I thought. I will update more reliable numbers later. Thanks and do not attack this!

[lllyasviel]

By the way, for old models, forge perfectly reproduce A1111 1.10 for all advanced features like prompt grammar, break, embeddings, loras, doras etc:

Forge:

Automatic 1111 original 1.10

[Mirabilis1729]

Hi @lllyasviel

One issue I have noticed is the 'Hires fix: show hires checkpoint' which looks like this in A1111:

Doesn't show up in Forge:

Even if you have it enabled in the uI settings.

Might not seem like an obvious need, but I've found it quite useful to be able to switch over to a different model at times in order to introduces a more stylistic look to outputs at the upscale stage and it would be very cool to see in Forge if possible.

[zethfoxster]

I too was hoping i could use it so i can switch from flux to an sd15 model and do basically img2img from main page to take advantage of fluxes extreme understanding and my million character loras lol

[Mirabilis1729]

I too was hoping i could use it so i can switch from flux to an sd15 model and do basically img2img from main page to take advantage of fluxes extreme understanding and my million character loras lol

Yeah it's a very cool feature that I don't think a lot of people either are aware of or understand, but it's really neat way of mixing things up.

[MitsuhaKitsune]

Thats a beautifull and totally unexpected (people think that forge are died) update, a lot of thanks @lllyasviel ^^

Now that we have a selector, I think it's good separate SD, XL and Flux models/loras depending on selected option (for example if I select XL only show XL/Pony checkpoints, loras, on the list and hide SD and Flux options in dropdowns).

If you consider it good idea, I don't have so much experience on gradio but I can try to do a PR for it.

[vihangasa14]

I have tested this sanity check in 6GB 3050 Vram and I took 2.5 mins

[Jonseed]

Hmm, I tried that prompt and seed on my 3060 12GB, using bnb-nf4 model, and this is what I get (took about 1.5 min). Quite a bit different... 🤔

[iamrohitanshu]

@Jonseed The only thing that seems to be different is your forge version.
Yours is Version: f2.0.1v1.10.1-previous-223-g86ee2d94
EDIT: might be the RNG.

[StanislawKarnacky]

What are your settings? Diffusion in Low Bits, Swap Method, Swap Location, GPU Weights (MB), could you share, please? In my case time goes up to 12+ minutes :(

[BerdoRod]

here is mine,

same prompt using:
NVIDIA GeForce RTX 2080 SUPER (Gigabyte) 8 Gbs
AMD Ryzen 9 3900X - RAM 32.0GB Dual-Channel DDR4 (17-17-17-39

[CCpt5]

You're ridiculously brilliant. Been following you coding run the past few days.....just updated. Really nothing to say - just other wordly skill/passion.

One thing I did want to ask (mention?) is that Ostris released a new LoRA training tool and I think there is some variation about the formatting of Flux LoRA. He used a formatting that required an update by Comfy earlier today. My first Flux LoRA works on Comfy atm but not on Forge 2.0. See his tweet here which links to the PR he submitted on ComyUI: https://x.com/ostrisai/status/1822367393555030487

Error is:

[LORA] LoRA version mismatch for KModel: d:\stable-diffusion-webui\models\Lora\Arnold-Person-LoRA-FLUX.safetensors███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:13<00:00,  1.51it/s]
Skipping unconditional conditioning when CFG = 1. Negative Prompts are ignored.
distilled_cfg_scale = 3.5
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:12<00:00,  1.57it/s]
Total progress: 100%|████████████████████████████████████████████████████████████████████

If this is an easy fix I'd really appreciate it. (And if it happens to be user error on my part please forgive me)

[E2GO]

Same problem using custom made LORA from Ostrisai

[rafiislambd]

please FLUX.1-schnell model convart to NF4... thanks

[Newtrial99]

imo converting this https://huggingface.co/drbaph/FLUX.1-schnell-dev-merged/tree/main to NF4 would be even better, it somewhat struggles with text compared to dev, but it has much better prompt adherence compared to Schnell imo, that too at 4 steps.

[peppepascale]

this would be really great. i second this

[MNeMoNiCuZ]

Converting FLUX.1-schnell original model is actually still more useful, since it has different licence than the Dev-model.

If you use the schnell-dev-merged, you get the dev-licence, which is not as friendly.

[DenOfEquity]

silveroxides has done this: https://huggingface.co/silveroxides/flux1-nf4-weights/blob/main/flux1-schnell-bnb-nf4.safetensors

While it's downloading, open up the webui install directory and go to webui\backend\huggingface\black-forest-labs and make a directory FLUX.1-schnell. Copy everything from the FLUX.1-dev directory into new directory.

[Swarzox]

imo converting this https://huggingface.co/drbaph/FLUX.1-schnell-dev-merged/tree/main to NF4 would be even better, it somewhat struggles with text compared to dev, but it has much better prompt adherence compared to Schnell imo, that too at 4 steps.

any update about this?

[Panchovix]

Hi there, many thanks for the work! Any chance to support FP16 Flux?

[Ling-APE]

Could this be implemented in Comfyui?

[Panchovix]

Pardon my ignorance, but doesn't Comfy already support flux? Or do you mean the optimizations?

[Ling-APE]

I think in this case it would be the ability to load nf4 weight in ComfyUI. And also the features like async, but I guess I should wait comfyUI's developers do that instead?

[Panchovix]

Comfy devs will probably check this thread and implement it shortly, but now their backend is different to the Forge backend, so they have to adapt the ability to use nf4.

[4lt3r3go]

should be already working with new nodes i saw somewere on reddit

[Pizzawookiee]

How would I convert an SDXL checkpoint to NF4 to avoid the 60s quantizing the model from fp16?

[Jonseed]

I don't know how to convert the checkpoint, but just selecting NF4 diffusion makes SDXL models about 35% faster for me. It doesn't replicate the same seed exactly though.

[noise2014]

what gpu u have ?

[Jonseed]

I have 3060 12GB.

[arcanite24]

Is this using T5?

[WingeD123]

comparing to the file size, T5 fp8 seems included in the safetensor

[mirek190]

T5xx 8 bit is really decreasing a picture quality from my tests . Easily noticeable hands deformation , etc

[FurkanGozukara]

Yes T5 FP16 better

[Jonseed]

The model card says "T5xxl in fp8e4m3fn" so it is using T5 fp8: https://huggingface.co/lllyasviel/flux1-dev-bnb-nf4

[gonewilds]

yeah they are using T5 fp8 which decrease the quality, just give it some time.. there will be fp16 version anyway

[mercantigo]

Hi guys, Any clue why I'm getting different result in my sanity check? 4090 in Stability Matrix here, xformers disabled.

[bews]

Same results on mine 4090:

Random number generator source GPU:

Random number generator source CPU:

Both different from the sanity check.

[greendra8]

Different on my 3070

[nickaubert]

I got exactly the same image as @HMRMike above on my 3090. As far as I can tell, the sanity check settings are all the same.

[f-rank]

Almost exact same as @bews with cpu generator. Tint on helmet reflection and some other small details seem diff though.
Still a bit diff from sanity check. On a 3070 8 GB and a 4060 16 GB.
I see small variations on others too. Dunno what am not doing or doing wrong.

[HMRMike]

I got exactly the same image as @HMRMike above on my 3090. As far as I can tell, the sanity check settings are all the same.

Yeah it seems the original check just used GPU RNG source so we cannot possibly recreate it with our different cards.
CPU setting recreates for example @bews's result perfectly. So it seems all is well

[WingeD123]

I see GOAT!

[sashasubbbb]

Is there a way to check if a GPU supports NF4? (2070 Super, to be exact.)

[JorgeR81]

@Mozer You may need to update your nvidia driver
I haven't tried NF4 yet, but my 1070 seems to have the required cuda version.
Last year, Windows 10 just updated my nvidia driver, without any warning !
I only noticed because my monitor was back for a second. 
But now it seems I have the latest CUDA version available around that time.

@kyism Did your times improve with NF4 ?
Have you tried Comfy UI ?

[kyism]

@JorgeR81 i only use forge.
nf4 is 4.6 times faster than fp8 with equal image quality.

[bli22]

In my 8GB 2070s, Forge work with fp4, 1024 x 1024 pixels, the pict be generate about 320 seconds. The result is approximate to comfyui with fp8

[unalkalkan]

Hey @kyism, how long does it take for you to generate 1024x1024 on your 1070?
I have a 1080 and I was not sure if it supported nf4, since it was trying to generate in 15+ mins.

Also what is your pytorch and cuda versions. I'm at Cuda 12.0 with torch==2.3.1. Also, my nvidia driver is a bit old (525.125.06 Linux), so I was suspecting that.

[kyism]

@unalkalkan torch231+cu121, win11, outdated pc(7years)

I won't use flux again until I upgrade to a 40 series :p

[lllyasviel]

Hi everyone, after tested with more devices, the speed-ups of bnb are more random than I thought. I will update more reliable numbers later. Thanks and do not attack this!

[DexterHuang]

That's an amazing update @lllyasviel
But there seems to be a bug when trying to use the new nf4 flux model

\webui\backend\attention.py", line 314, in attention_xformers mask_out[:, :, :mask.shape[-1]] = mask RuntimeError: The expanded size of the tensor (1) must match the existing size (64) at non-singleton dimension 0. Target sizes: [1, 256, 256]. Tensor sizes: [64, 256, 256] The expanded size of the tensor (1) must match the existing size (64) at non-singleton dimension 0. Target sizes: [1, 256, 256]. Tensor sizes: [64, 256, 256]

I have used --xformer in command arg, if I don't it will crash when I try to start, not sure why as well.. below is error when I don't use --xformer:
D:\workplace\webui_forge_cu121_torch21\system\python\lib\site-packages\transformers\utils\hub.py:127: FutureWarning: Using TRANSFORMERS_CACHEis deprecated and will be removed in v5 of Transformers. UseHF_HOME instead. warnings.warn( Traceback (most recent call last): File "D:\workplace\webui_forge_cu121_torch21\webui\launch.py", line 51, in <module> main() File "D:\workplace\webui_forge_cu121_torch21\webui\launch.py", line 47, in main start() File "D:\workplace\webui_forge_cu121_torch21\webui\modules\launch_utils.py", line 544, in start import webui File "D:\workplace\webui_forge_cu121_torch21\webui\webui.py", line 21, in <module> initialize.imports() File "D:\workplace\webui_forge_cu121_torch21\webui\modules\initialize.py", line 32, in imports from modules import processing, gradio_extensions, ui # noqa: F401 File "D:\workplace\webui_forge_cu121_torch21\webui\modules\processing.py", line 19, in <module> from modules import devices, prompt_parser, masking, sd_samplers, lowvram, infotext_utils, extra_networks, sd_vae_approx, scripts, sd_samplers_common, sd_unet, errors, rng, profiling File "D:\workplace\webui_forge_cu121_torch21\webui\modules\sd_samplers.py", line 5, in <module> from modules import sd_samplers_kdiffusion, sd_samplers_timesteps, sd_samplers_lcm, shared, sd_samplers_common, sd_schedulers File "D:\workplace\webui_forge_cu121_torch21\webui\modules\sd_samplers_kdiffusion.py", line 5, in <module> from modules import sd_samplers_common, sd_samplers_extra, sd_samplers_cfg_denoiser, sd_schedulers, devices File "D:\workplace\webui_forge_cu121_torch21\webui\modules\sd_samplers_common.py", line 6, in <module> from modules import devices, images, sd_vae_approx, sd_samplers, sd_vae_taesd, shared, sd_models File "D:\workplace\webui_forge_cu121_torch21\webui\modules\sd_models.py", line 21, in <module> from backend.loader import forge_loader File "D:\workplace\webui_forge_cu121_torch21\webui\backend\loader.py", line 16, in <module> from backend.nn.vae import IntegratedAutoencoderKL File "D:\workplace\webui_forge_cu121_torch21\webui\backend\nn\vae.py", line 3, in <module> from backend.attention import attention_function_single_head_spatial File "D:\workplace\webui_forge_cu121_torch21\webui\backend\attention.py", line 12, in <module> import xformers ModuleNotFoundError: import of xformers halted; None in sys.modules

[navytut2]

Facing exactly same error as above, though with FP8

Edit: Solved it by adding --disable-xformers to webui file. Working fine now.

[DaveScream]

Hi everyone, after tested with more devices, the speed-ups of bnb are more random than I thought. I will update more reliable numbers later. Thanks and do not attack this!

Do you have plans to make TensorRT version of NF4?

[IndigoDosSantos]

Great update, thanks a lot!

Inpainting/soft-inpainting with Flux doesn't work for me yet.

[IndigoDosSantos]

Both are supported now, thanks a lot, @lllyasviel ! 💖

[1onica1]

how?

[ivanoff13]

img2img does not work((

[IndigoDosSantos]

It is supported now!

[Newtrial99]

Thanks a lot for your hard work, would it be possible for you to convert this to NF4 as well ? https://huggingface.co/drbaph/FLUX.1-schnell-dev-merged/tree/main pls, as it can do decent quality better than schnell in 4 steps

[Swarzox]

would be amazing

[ViratX]

Bump!

[AlexXingRob]

@lllyasviel Can "flux1-dev-bnb-nf4-v2" be used with SimpleTuner to train a FLUX.1 LoRA?

[Satochiju]

Hi illya, just to thank you and tell you that Spaces is great, but what you've done with Diffusion in Low Bits is amazing, my 8G VRAM card no longer suffers hahaha

[Mark999a]

Is it any way to FLUSH SHARED GPU MEMORY? Any button or setting? Instead of restarting forge? Even if i change "swap location" it still loaded and eat all RAM.

[yamfun]

does the prompt (weight:1.4) syntax work for Flux?

[HMRMike]

It does, but the effect is not very strong. It seems more effective for values less than 1, but that's probably the opposite of what we want in most cases. Coupled with a suitable LORA it seems to be more potent.

[kenny00968]

C:\StableDiffusion\stable-diffusion-webui-forge\system\python\lib\site-packages\transformers\utils\hub.py:127: FutureWarning: Using TRANSFORMERS_CACHE is deprecated and will be removed in v5 of Transformers. Use HF_HOME instead.
warnings.warn(
这个警告怎么解决？

[geroldmeisinger]

How to Convert a FLUX-Dev Checkpoint to an NF4 Model?

[livineternity]

Hi, im on a macbook m3 18GPU metal 3
when trying to generate an image with flux I get "RuntimeError: linear(): input and weight.T shapes cannot be multiplied (961x64 and 1x98304)
Time taken: 47.6 sec."
I downloaded the nf4 model so far. now trying to download the older dev fp8. Im new to forge. can anyone help?

[livineternity]

now I read I can use only the fp8 model on my Mac (18gpu, m3 pro) but I get this error now: "TypeError: Trying to convert Float8_e4m3fn to the MPS backend but it does not have support for that dtype."

:|

[yamfun]

it seems the [A:B:10] syntax does not work for Flux

[btaskel]

Hello, how should I use shared memory in Comfyui?🤔

[edulopesf]

Running on a RTX 3050M 4G VRAM.... Sanity check @ 5 min on 1st run, 2nd run @ 3 min.

[LeonSun128]

You send email has been received .(邮件已接收)                                                                  Leon.Sun(亮)

[aearone]

3060ti 8 gb vram + 32 gb ram, generation time of 768x768 with flux1-dev-bnb-nf4-v2 model and 20 steps takes about 40 seconds, and with 30 steps - 1 minute 10 seconds. If I try to load the full flux1-dev model on 24 GB + ae.vae + clip_l + t5xxl_fp16 then I get a very, very long generation time, from 10 minutes and sometimes it may not start at all, switching to eternal loading. What could be the reason?

[dan4ik94]

Lack of RAM. In total you try to load ~34 GB of stuff, that's huge.

[JollyRoger112]

How are you getting fast generation times? I have a laptop 3070 8GB + 32GB RAM, and I'm getting 1 image in 5 minutes.

[JoeGeeGameDev]

What other checkpoints you recommend? I only have a 3060 12g

[MitsuhaKitsune]

I have the same card (ASUS Dual 3060 12GB) and I can use Flux, check the NF4 guide, gona take a bit more than XL to generate, but you can.

[dan4ik94]

I'm mainly using dev with 6 or 8 steps loras. 3060 owner too.

[DrakeThePM]

I cannot seem to figure out what my issue is. My specs are:
RTX 4090, i9 13900k, 64gb of RAM

In general my generations take a very long time compared to others with the same spec, anywhere from 40 seconds to a minute and a half at 1024x1024. Over time I will eventually get a CUDA error where all my memory is being used even after the CUDA error until I close Forge.

Ignore the pixelwave model, I just tried switching it at the end, this was run on dev.

[dan4ik94]

I have a 3060 but I see some possible issues on your side:

• diffusion in low bits should be automatic fp16
• swap location should be CPU
• try lowering the GPU weights
• make sure system fallback is disabled in your nvidia control panel
• try a 11GB fp8 model to see if there is a difference in speed

[ETCore7]

Amazing work, thank you very much.