Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Web-only video support #7261

Merged
merged 49 commits into from
Aug 29, 2024
Merged
Changes from 1 commit
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
1d007e6
temp
jprochazk Aug 15, 2024
926473d
video archetype
jprochazk Aug 19, 2024
928e741
update todo
jprochazk Aug 19, 2024
369fa7c
wip
jprochazk Aug 20, 2024
6d3665c
Merge branch 'main' into jan/unbox-mp4
jprochazk Aug 21, 2024
0614020
update lockfile
jprochazk Aug 21, 2024
33376ea
stuff!
jprochazk Aug 23, 2024
db86e15
Merge branch 'main' into jan/unbox-mp4
jprochazk Aug 23, 2024
00a11ab
fix patch
jprochazk Aug 23, 2024
ca08206
use `partition_point`
jprochazk Aug 27, 2024
63b0680
warn_once
jprochazk Aug 27, 2024
84f22bc
update video load error
jprochazk Aug 27, 2024
caf9ec4
return zeroed texture for negative timestamps
jprochazk Aug 27, 2024
cb9be60
Merge remote-tracking branch 'origin/main' into jan/unbox-mp4
jprochazk Aug 27, 2024
0d72ace
mention `AssetVideo` in `gen_common_index.py`
jprochazk Aug 27, 2024
f984a64
fix lints + add UnsupportedCodec error
jprochazk Aug 27, 2024
0c50535
fix lints
jprochazk Aug 27, 2024
54eb15a
fix lints
jprochazk Aug 27, 2024
8f0fd10
fix lints
jprochazk Aug 27, 2024
827405f
fix lints
jprochazk Aug 27, 2024
fbfa6da
Merge branch 'main' into jan/unbox-mp4
jprochazk Aug 28, 2024
7927aac
add "experimental" marker
jprochazk Aug 28, 2024
4bd2f17
dont use emoji in cpp docs
jprochazk Aug 28, 2024
3176c91
Add support for loading videos directly from file
emilk Aug 28, 2024
a95811f
update wording
jprochazk Aug 29, 2024
8c5f950
wording
jprochazk Aug 29, 2024
9a4b57e
`from_file` -> `from_file_path`
jprochazk Aug 29, 2024
0ee008f
better `Debug` impl for `TimeMs`
jprochazk Aug 29, 2024
dff6823
remove comment
jprochazk Aug 29, 2024
de99b07
sort
jprochazk Aug 29, 2024
1fdf89b
fix
jprochazk Aug 29, 2024
ea5c048
docs
jprochazk Aug 29, 2024
b17a816
some refactoring
jprochazk Aug 29, 2024
da7ca36
docs
jprochazk Aug 29, 2024
6275471
todo
jprochazk Aug 29, 2024
e008095
warn on native
jprochazk Aug 29, 2024
d6f6996
restructure
jprochazk Aug 29, 2024
d1efef9
fix
jprochazk Aug 29, 2024
5ab19ab
remove temp
jprochazk Aug 29, 2024
ff38ade
fix snippet
jprochazk Aug 29, 2024
755153f
explain
jprochazk Aug 29, 2024
1565a8c
fix
jprochazk Aug 29, 2024
0008aed
oops
jprochazk Aug 29, 2024
9f4b5d2
doc
jprochazk Aug 29, 2024
dd5682a
Merge branch 'main' into jan/unbox-mp4
jprochazk Aug 29, 2024
fa1e64e
undo settings change
jprochazk Aug 29, 2024
6e9fa65
remove usage of `TimeKey`
jprochazk Aug 29, 2024
4571be6
fix compile
jprochazk Aug 29, 2024
a202f1b
remove video example
jprochazk Aug 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
return zeroed texture for negative timestamps
  • Loading branch information
jprochazk committed Aug 27, 2024
commit caf9ec418bb07dd0f6cfc51d2f17127f39c9c55d
22 changes: 22 additions & 0 deletions crates/viewer/re_renderer/src/renderer/video.rs
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ impl Video {
self.decoder.height()
}

/// Returns a texture with the latest frame at the given timestamp.
///
/// If the timestamp is negative, a zeroed texture is returned.
jprochazk marked this conversation as resolved.
Show resolved Hide resolved
pub fn get_frame(&mut self, timestamp_s: f64) -> GpuTexture2D {
re_tracing::profile_function!();
self.decoder.get_frame(TimeMs::new(timestamp_s * 1e3))
Expand Down Expand Up @@ -178,6 +181,7 @@ mod decoder {
device: Arc<wgpu::Device>,
queue: Arc<wgpu::Queue>,
texture: GpuTexture2D,
zeroed_texture_float: GpuTexture2D,

decoder: web_sys::VideoDecoder,

Expand Down Expand Up @@ -216,18 +220,27 @@ mod decoder {
}
})?;

// NOTE: both textures are assumed to be rgba8unorm
let texture = super::alloc_video_frame_texture(
&device,
&render_context.gpu_resources.textures,
data.config.coded_width as u32,
data.config.coded_height as u32,
);
let zeroed_texture_float = GpuTexture2D::new(
render_context
.texture_manager_2d
.zeroed_texture_float()
.clone(),
)
.expect("expected texture to be 2d");

let mut this = Self {
data,
device,
queue,
texture,
zeroed_texture_float,

decoder,

Expand Down Expand Up @@ -257,6 +270,11 @@ mod decoder {
}

pub fn get_frame(&mut self, timestamp: TimeMs) -> GpuTexture2D {
if timestamp.as_f64() < 0 {
// TODO(andreas): This is a hack, we should have a better way to handle this
return self.zeroed_texture_float.clone();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning None seems like maybe a better choice?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to always return some texture, even if it's going to be empty. If we need to know if a given frame is available immediately, we can add a separate method for that that would just look inside the frames buffer.

}

let Some(segment_idx) =
latest_at_idx(&self.data.segments, |segment| segment.timestamp, &timestamp)
else {
Expand Down Expand Up @@ -470,6 +488,10 @@ mod decoder {
use crate::resource_managers::GpuTexture2D;
use crate::RenderContext;

// TODO(jan): remove once we have native video decoding
#[allow(unused_imports)]
use super::latest_at_idx;

use re_video::TimeMs;

use super::alloc_video_frame_texture;
Expand Down
Loading