Skip to content

Quickstart

We are going to start from a simple example with the scene data/blender/lego , where a 3D Gaussian Splatting model will be trained, tested, exported and visualized. You can find code from RadianceFieldStudio/examples/quickstart .

1. Data Preparation

Download NeRF Synthetic Dataset from HuggingFace Repository maintained by NerfBaselines .

cd RadianceFieldStudio
python scripts/data/download.py blender --scene lego --use-proxy # Use proxy for Chinese users

You will get Lego now: data/blender/lego .

3. Train a 3DGS

The script examples/quickstart/01_train_3dgs.py provides code for 3DGS training:

examples/quickstart/01_train_3dgs.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
from rfstudio.data import MultiViewDataset
from rfstudio.engine.train import Experiment, TrainTask
from rfstudio.model import GSplatter
from rfstudio.trainer import GSplatTrainer

std_train_task = TrainTask(
    dataset=MultiViewDataset(
        path=...,
    ),
    model=GSplatter(
        background_color='black',
        sh_degree=0
    ),
    experiment=Experiment(name=...),
    trainer=GSplatTrainer(
        num_steps=30000,
        batch_size=1,
        num_steps_per_val=300,
        num_steps_per_save=6000,
        mixed_precision=False
    ),
    cuda=0,
    seed=1
)

if __name__ == '__main__':
    std_train_task.run()

Run the script in your terminal and you can watch progress from the terminal:

python train.py \
    --dataset.path data/blender/lego \
    --experiment.name test

4. Check Training Results

After training for ~5min, the task will be done and you can use Ctrl+C to exit.

The related training results will be dumped at outputs/test/${timestamp} :

  • Checkpoints: Model weights are saved at outputs/test/${timestamp}/ckpts .

  • Validation: novel views as well as their ground truth are rendered at outputs/test/${timestamp}/dump/val .

  • Log: training details will be logged at outputs/test/${timestamp}/log.txt .

  • Arguments: training arguments will be automatically generated at outputs/test/${timestamp}/task.py . (1)

    1. It enables you to rerun an experiment with the completely same arguments, simply by:

      python outputs/test/${timestamp}/task.py

5. Write Your First Script

In Section 3, we have run an existing train script. Next, we are now going to write our own script from scratch.

5.1 Hello World

Create a new script named test.py and start with Hello World :

1
2
3
4
5
def run():
    print('Hello World!')

if __name__ == '__main__':
    run()

$ python test.py
Hello World!

5.2 Start From Task

The above script succeed in executing the main function run . Similarly, Radiance Field Studio abstracts any executable script into a Task class, which always has a run method as the entry point of the script. Users need to import the Task class as a base class and override its run method through inheritance:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
from dataclasses import dataclass
from rfstudio.engine.task import Task


@dataclass                     # dataclass decorator is a must
class Tester(Task):
    def run(self) -> None:
        print('Hello World!')


if __name__ == '__main__':
    tester = Tester()
    tester.run()

$ python test.py
Hello World!

5.3 CLI Support

One of the advantages of doing this is that Radiance Field Studio provides convenient command-line support based on tyro. The dataclass decorator is used to add fields to this class, after which these fields will be automatically recognized:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
from dataclasses import dataclass
from rfstudio.engine.task import Task


@dataclass                     # dataclass decorator is a must
class Tester(Task):

    num_a: int = ...           # ellipsis means "required"
    num_b: int = 6             # default value is 6
    num_c: int = ...           # ellipsis means "required"
    info: str = 'Hello World!' # default value is 'Hello World!'

    def run(self) -> None:
        print(self.info)       # use self.${field} to access any field
        print(f'{self.num_a + self.num_b + self.num_c = }')


if __name__ == '__main__':
    tester = Tester()
    tester.run()

$ python test.py -h
usage: test.py [-h] [OPTIONS]

╭─ options ─────────────────────────────────────────────────────────────────────────╮
│ -h, --help              show this help message and exit                           │
│ --seed {None}|INT       (default: None)                                           │
│ --cuda {None}|INT       (default: None)                                           │
│ --num-a INT             ellipsis means "required" (required)                      │
│ --num-b INT             default value is 6 (default: 6)                           │
│ --num-c INT             ellipsis means "required" (required)                      │
│ --info STR              default value is 'Hello World!' (default: 'Hello World!') │
╰───────────────────────────────────────────────────────────────────────────────────╯
$ python test.py --num-a 1 --num-c 3
Hello World!
self.num_a + self.num_b + self.num_c = 10
$ python test.py --num-a 1 --num-c 3 --num-b 9 --info "Good Morning\!"
Good Morning!
self.num_a + self.num_b + self.num_c = 13

5.4 Edit Argument

Before we call run function to execute the task, it is possible for us to explicitly initialize the arguments or dynamically modify them:

18
19
20
21
if __name__ == '__main__':
    tester = Tester(num_c=9, info=...)
    tester.num_b = 7
    tester.run()

$ python test.py -h
usage: test.py [-h] [OPTIONS]

╭─ options ──────────────────────────────────────────────────────────╮
│ -h, --help              show this help message and exit            │
│ --seed {None}|INT       (default: None)                            │
│ --cuda {None}|INT       (default: None)                            │
│ --num-a INT             ellipsis means "required" (required)       │
│ --num-b INT             default value is 6 (default: 7)            │
│ --num-c INT             ellipsis means "required" (default: 9)     │
│ --info STR              default value is 'Hello World!' (required) │
╰────────────────────────────────────────────────────────────────────╯

5.5 Serialization & Deserialization

Radiance Field Studio provides APIs for serialization and deserialization of Task. For example, the class method TrainTask.load_from_script is used to deserialize Task object from outputs/test/${timestamp}/task.py, which is automatically generated in Section 3 . We can use the API to build a simple tester.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
from dataclasses import dataclass
from pathlib import Path

from rfstudio.engine.task import Task
from rfstudio.engine.train import TrainTask
from rfstudio.model import GSplatter


@dataclass
class Load3DGS(Task):

    load: Path = ...

    def run(self) -> None:
        train_task = TrainTask.load_from_script(self.load)
        model = train_task.model
        assert isinstance(model, GSplatter)
        print(f'Result: #Gaussians = {model.gaussians.shape[0]}')


if __name__ == '__main__':
    Load3DGS(cuda=0).run()

$ python test.py --load ./outputs/test/${timestamp}/task.py
Result: #Gaussians = 116590

6. Test Your Model

Rewrite test.py for pretrained model testing: (1)

  1. You can also find the code at RadianceFieldStudio/examples/quickstart/03_test_3dgs.py .
test.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
from dataclasses import dataclass
from pathlib import Path

import torch

from rfstudio.engine.experiment import Experiment
from rfstudio.engine.task import Task
from rfstudio.engine.train import TrainTask
from rfstudio.model import GSplatter


@dataclass
class Test3DGS(Task):

    load: Path = ...

    experiment: Experiment = Experiment(
        name='test_3dgs',
        output_dir=Path('exports')
    )

    def run(self) -> None:
        train_task = TrainTask.load_from_script(self.load)
        model = train_task.model
        assert isinstance(model, GSplatter)
        dataset = train_task.dataset
        trainer = train_task.trainer

        with torch.no_grad():
            model.train(False)
            losses = []
            psnrs = []
            for inputs, gt_outputs, indices in dataset.get_train_iter(1, shuffle=False, infinite=False):
                loss, metrics, visualization = trainer.step(
                    model,
                    inputs,
                    gt_outputs,
                    training=False,
                    visual=True
                )
                self.experiment.dump_image('train-view', index=indices[0], image=visualization)
                losses.append(loss.item())
                psnrs.append(metrics['psnr'].item())
            print("Eval: loss={:.5f} psnr={:.3f}".format(
                sum(losses) / len(losses),
                sum(psnrs) / len(psnrs)
            ))

if __name__ == '__main__':
    Test3DGS(cuda=0).run()

Run the script and the results are:

Eval: loss=0.01009 psnr=33.638

7. Visualize 3DGS

Create another script named visualize.py for visualization: (1)

  1. You can also find the code at RadianceFieldStudio/examples/quickstart/04_visualize_3dgs.py .
visualize.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
from dataclasses import dataclass
from pathlib import Path

from rfstudio.engine.task import Task
from rfstudio.engine.train import TrainTask
from rfstudio.model import GSplatter
from rfstudio.visualization import vis_3dgs


@dataclass
class Vis3DGS(Task):

    load: Path = ...

    port: int = ...

    def run(self) -> None:
        train_task = TrainTask.load_from_script(self.load)
        model = train_task.model
        assert isinstance(model, GSplatter)
        vis_3dgs(model, port=self.port)


if __name__ == '__main__':
    Vis3DGS(port=6789).run()

Then, you can visualize 3DGS in the interactive web viewer.