Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
1.2k views
in Technique[技术] by (71.8m points)

io - Reading file as continous stream using Rust

I am writing a rust program that attach to multiple logs file and process elements as soon as new rows are available.

I cannot find any convenient way to read a file line by line in rust std lib and do not stop when we reach EOF.

I have try the bellow solutions:

Method 1: using stdin

use std::io::{self, BufRead};
use std::str::FromStr;

fn main() {
    let stdin = io::stdin();
    for line in stdin.lock().lines() {
        let line = line.unwrap();
        // Process line
        // ...
    }
}

Drawbacks:

  • Cannot process multiple files
  • Need to use shell to redirect IO
  • stdin is not available anymore

Method 2: using BufRead::read_line

use std::fs::File;
use std::io::{self, BufRead, BufReader};
use std::thread;
use std::time::Duration;

fn main() -> Result<(), io::Error> {
    let filename = "test.log";
    let file = File::open(&filename)?;

    let mut reader = BufReader::new(file);
    let mut line = String::new();

    loop {
        match reader.read_line(&mut line) {
            Ok(bytes_read) if bytes_read == 0 => {
                // Be nice with CPU
                thread::sleep(Duration::from_millis(50));
            }
            Ok(_) => {
                // Process line
                // ...

                // Clear content
                line.clear();
            }
            Err(err) => {
                return Err(err);
            }
        };
    }
}

Drawbacks:

  • Need to have a thread::sleep to avoid using 100% CPU
  • Code is less clear than the stream approach

Is there any way to read a file as infinite stream using rust std lib ?

Is there some crates available to do this kind of processing ?

question from:https://stackoverflow.com/questions/65858731/reading-file-as-continous-stream-using-rust

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)
Waitting for answers

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...