agate, util: fix varint decode and add header #29
Conversation
Signed-off-by: Alex Chi <[email protected]>
Signed-off-by: Alex Chi <[email protected]>
Signed-off-by: Alex Chi <[email protected]>
Signed-off-by: Alex Chi <[email protected]>
skyzh
changed the title
src/util.rs
Outdated
| c + 1 | ||
| } | ||
|
|
||
| pub unsafe fn decode_varint_uncheck(bytes: &[u8]) -> (u64, usize) { |
There was a problem hiding this comment.
You can write it in safe rust by using for loop.
https://github.com/BusyJay/pecan/blob/d4a17231253cd6b304a15d9d0f65edd5a0bcceab/utils/src/codec.rs#L54-L84 is an example.
There was a problem hiding this comment.
I copied codec.rs and modified input parameter as BytesMut, plus adapting error type to agatedb.
| let encoded_len = self.encoded_len(); | ||
| bytes.reserve(encoded_len); | ||
| let read = unsafe { | ||
| let buf = bytes.bytes_mut(); |
There was a problem hiding this comment.
Perhaps adding an if check before entering get_unchecked_mut is better.
There was a problem hiding this comment.
I've added the check assert!(buf.len() >= encoded_len);. Seems that reserve will always ensure the capacity is enough, I'm not sure whether this length check is necessary or not.
Signed-off-by: Alex Chi <[email protected]>
Signed-off-by: Alex Chi <[email protected]>
Signed-off-by: Alex Chi <[email protected]>
| @@ -1,10 +1,70 @@ | |||
| use super::Result; | |||
| use crate::util::binary::{ | |||
There was a problem hiding this comment.
How about using a crate like https://docs.rs/integer-encoding/2.1.1/integer_encoding/ instead of porting a go binary. /cc @BusyJay
There was a problem hiding this comment.
The latest patch seems to copy the implementation from a library written by me. Only three functions are necessary and they are quite simple. I suggest to avoid adding new dependencies and delete unused functions in the futures.
There was a problem hiding this comment.
I'll remove zig-zag functions and signed integer functions from binary.rs
There was a problem hiding this comment.
Done. Now we only retain functions for varint u32 and u64.
Signed-off-by: Alex Chi <[email protected]>
Signed-off-by: Alex Chi <[email protected]>
Signed-off-by: Alex Chi <[email protected]>
Signed-off-by: Alex Chi <[email protected]>
|
@Connor1996 Now that |
The implementation of varint decode seems to be wrong, and I've rewritten it from the Golang version.
Encode / decode has been moved to
util.rs.By the way, I also implemented decode / encode of WAL header.