Skip to content

buffered io input for fast read line - #82

Open
c-cube wants to merge 7 commits into
ygrek:masterfrom
c-cube:sc/buffered-io-input-for-fast-read-line
Open

c-cube wants to merge 7 commits into
ygrek:masterfrom
c-cube:sc/buffered-io-input-for-fast-read-line

Conversation

@c-cube

@c-cube c-cube commented Oct 26, 2026

Copy link
Copy Markdown

This is related to performance work by alexander goryanets. My goal is to enable read_line to be actually fast for us, without giving up on the IO.input abstraction.

AI disclaimer: claude reviewed my code, and wrote tests. I wrote the src/ code.

the goal is to significantly accelerate `read_line` when this is
available.

A buf in \r handling in existing read_line is also fixed at the same
time.
@c-cube
c-cube force-pushed the sc/buffered-io-input-for-fast-read-line branch from 0fd78db to 90f29cb Compare October 26, 2026 20:08
Comment thread src/IO.ml
as_buffered;
}

let create_in ~read ~input ~close : input = create_in_full ~read ~input ~close ()

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the point of having separate create_in_full? I can imagine backwards-compatibility in the strictest sense - signature mismatch for reimplementation of IO module - but new function breaks it same way

Comment thread src/IO.ml
);
in_close = i.in_close
in_close = i.in_close;
as_buffered = (fun () -> None);

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this pass through buffer from i?

Comment thread src/IO.ml
| '\n' ->
()
| '\r' ->
if !cr then Buffer.add_char b '\r';

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a change in behaviour? it seems it fixes swallowing multiple \r at the end of line?

Comment thread src/IO.mli
- the view is shared with the owning {!input}: [read]/[input] on that
input and operations on the view consume from the same data.

Use {!create} rather than building the record by hand, as it

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can use private to enforce this but i guess it is moot given the fields are mutable

Comment thread src/IO.mli
into [buf], sets [pos<-0], and [len<-num_of_bytes_refilled].
When len>0 it may do nothing, or read more data. *)
}
(** View of {!input} with exposed buffer, only for some streams. Providing

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i wondered if it is better to expose more functional api but then i realized stdlib still doesn't have substring type that we could mimic..

Comment thread src/IO.ml
while not !found && !i < last do
if Bytes.unsafe_get input.buf !i = c then found := true else incr i
done;
if !found then !i else -1

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

String.find_from + compare with len?

Comment thread src/IO.ml
cr := ends_in_cr input len;
let s = Bytes.sub_string input.buf input.pos (skip_last_if_cr len) in
Buf_in.consume input (len+1);
s

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fat chance i can review this with any confidence 😅

Comment thread src/IO.ml
if buf_in.len = 0 then raise No_more_input;
let c = Bytes.unsafe_get s buf_in.pos in
buf_in.pos <- buf_in.pos + 1;
buf_in.len <- buf_in.len - 1;

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consume?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants