首页 文章

Webflux FileUpload Minio

提问于
浏览 次
0

我试图弄清楚为什么通过WebFlux endpoints 上传到minio（符合S3的文档存储）的文件没有完成;我总是只有4kb的文件到Minio .

我的终点：

public Mono<ServerResponse> uploadFile(ServerRequest request) {
        log.info("Uploading file...");
        log.info("Content Type: {}", request.headers().contentType().orElse(MediaType.TEXT_PLAIN));

        return request.body(BodyExtractors.toMultipartData())
            .flatMap(map -> {
                Map<String, Part> parts = map.toSingleValueMap();
                return Mono.just((FilePart) parts.get("files"));
            })
            .flatMap(this::saveFile)
            .flatMap(part -> ServerResponse.ok().body(BodyInserters.fromObject(part)));
    }

    private Mono<String> saveFile(FilePart part) {
        return part.content().map(dataBuffer -> {
            try {
                log.info("Putting file {} to minio...", part.filename());
                client.putObject("files", part.filename(), dataBuffer.asInputStream(), part.headers().getContentType().getType());
            } catch(Exception e) {
                log.error("Error storing file to minio", e);
                return part.filename();
            }
        return part.filename();
        }).next();
    }

我很确定这是一个阻塞与非阻塞问题，但如果我尝试添加一个 blockFirst() 调用，我会得到一个Exception，说它在运行时是不允许的 .

有没有办法有效地传输数据，或者这是Minio客户端与WebFlux不兼容的情况？

我试图从这样的React组件发布：

class DataUpload extends React.Component {
    constructor(props) {
        super(props);

        this.state = {
            fileURL: '',
        };

        this.handleUploadData = this.handleUploadData.bind(this);
    }

    handleUploadData = ev => {
        ev.preventDefault()

        const data = new FormData();
        data.append('files', this.uploadInput.files[0]);
        data.append('filename', this.fileName.value);
        fetch('http://localhost:8080/requestor/upload', {
            method: 'POST',
            body: data,
        }).then((response) => {
            response.json().then((body) => {
                this.setState({ fileURL: `http://localhost:8080/${body.file}`});
            });
        });
    }

    render() {
        return (
            <form onSubmit={this.handleUploadData}>
                <div>
                    <input ref={(ref) => { this.uploadInput = ref; }} type="file" />
                </div>
                <div>
                    <input ref={(ref) => { this.fileName = ref; }} type="text" placeholder="Enter the desired name of the file" />
                </div>
                

                <div><button>Upload</button></div>
            </form>
        );
    }
}

export default DataUpload;
reactjs spring-webflux asyncfileupload minio

2 回答

  • 0

    我总是只将4kb的文件输入Minio

    那是因为 Spring 天的大块文件变成4kb的部分，你必须自己收集它们 . 你可以这样做：

    request.body(BodyExtractors.toMultipartData())
    .map(dataBuffers -> dataBuffers.get("files"))
    .filter(Objects::nonNull)
    //get the file name and pair it with it's "Part"
    .map(partsList -> {
        List<Pair<String, Part>> pairedList = new ArrayList<>();

        for (Part part : partsList) {
            String fileName = ((FilePart) part).filename();
            pairedList.add(new Pair<>(fileName, part));
        }

        return pairedList;
    })
    .flux()
    .flatMap(Flux::fromIterable)
    //here we collect all of the file parts with the buffer operator and zip them with filename
    .flatMap(partWithName -> Mono.zip(Mono.just(partWithName.getFirst()), partWithName.getSecond().content().buffer().single()))
    .buffer()
    .single()
    .doOnNext(filePartsWithNames -> {
        //here we have a list of all uploading file parts and their names
        for (Tuple2<String, List<DataBuffer>> filePartsWithName : filePartsWithNames) {
            String fileName = filePartsWithName.getT1();
            List<DataBuffer> buffers = filePartsWithName.getT2();

            System.out.println("Filename = " + fileName);

            //"buffers" is a list of 4kb chunks of the files
            for (DataBuffer buffer : buffers) {
                System.out.println("Buffer size = " + buffer.readableByteCount());

                //here you can use buffer.asInputStream() to read the file part and
                //then save it on disk or do something else with it
            }
        }
    })
    回复于
  • 0

    对于未来的读者，

    我在封装 org.springframework.core.io.buffer 中使用 DataBufferUtils.join() 之前通过加入dataBuffers来解决这个问题，然后再将其暴露为inputStream .

    Mono<ServerResponse> uploadSingleImageToS3(ServerRequest request) {
    return request.body(toMultipartData())
            .flatMap(parts -> {
                Map<String, Part> part = parts.toSingleValueMap();
                return Mono.just((FilePart) part.get("file"));
            })
            .flatMap(this::uploadToS3Bucket)
            .flatMap(filename -> ServerResponse.ok().body(fromObject(filename)));
}

private Mono<String> uploadToS3Bucket(FilePart part) {
    return DataBufferUtils.join(part.content())
            .map(dataBuffer -> {
                String filename = UUID.randomUUID().toString();
                log.info("filename : {}", filename);

                ObjectMetadata metadata = new ObjectMetadata();
                metadata.setContentLength(dataBuffer.capacity());
                PutObjectRequest putObjectRequest = new PutObjectRequest(answerImagesBucket, filename, dataBuffer.asInputStream(), metadata);
                transferManager.upload(putObjectRequest);

                return filename;
            });
}
    回复于

相关问题