db_bench core dumped with current master

[wenhaocs]

Hi, I was trying to run db_bench and test the performance with S3, core dumped at CreateAwsEnv:

db_bench_tool.cc:1527: rocksdb::Env* CreateAwsEnv(const string&, std::unique_ptr<rocksdb::Env>*): Assertion st.ok()' failed`

Steps to reproduce the behavior

OS: Fedora release 35
GCC: 11.2.1 20211203 (Red Hat 11.2.1-7)
cmake: 3.22.1
aws-sdk-c++: master(1.9.217)

Build aws-sdk-c++:

cmake .. -DENABLE_TESTING=OFF -DAUTORUN_UNIT_TESTS=OFF -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DCMAKE_CXX_FLAGS="-Wno-deprecated-declarations"
make install

Build rocksdb-cloud

cmake .. -DWITH_AWS=1 -DFAIL_ON_WARNINGS=OFF -DWITH_TESTS=OFF  -DWITH_GFLAGS=ON -DWITH_SNAPPY=ON -DWITH_ZSTD=ON  -DWITH_ZLIB=ON  -DWITH_LZ4=ON -DWITH_BZ2=ON
make

Run db_bench
./db_bench --env_uri="s3://" --aws_access_id="xxx" --aws_secret_key="xxx" --aws_region="us-west-2" --keep_local_sst_files --benchmarks="fillseq" --db=/data/rocksdb_main --duration=30 --num=10000000

Any lights on the issue? Thanks!

[moody6520]

Same here, did you solve the problem? Or any one can help?

[wenhaocs]

But I tried the old version and got my POC data (sorry forgot about the exact version). Not sure whether the current master works...

[dimu637]

@wenhaocs thx for the reply, Do you mean the old version for aws-sdk-c++ or the old version of this rocksdb-clould repo?

[wenhaocs]

both

[JacksonWang6]

i choose modify dbbench, and dont use its CreateAwsEnv function, instead of my own implemention to open the DBCloud

[AntiO2]

When using db_bench in v6.7.3, I encountered the same issue. This issue troubled me for an entire day, but the solution is actually very simple and doesn't require modifying the code.

I used st.ToString() to print the information, and it displayed Invalid argument: Must specify both src bucket name and path. During debugging, I found that the two properties (object_ and bucket_) of coptions.src_bucket were empty.

---

My solution is as follows:

Export the following three environment variables before the benchmark test:

export ROCKSDB_CLOUD_BUCKET_NAME="s3-antio2" # The name of the bucket used by your benchmark
export ROCKSDB_CLOUD_BUCKET_PREFIX="test." #  The final bucket used is prefix + name
# export ROCKSDB_CLOUD_BUCKET_PREFIX="" # It is OK to leave prefix empty
export ROCKSDB_CLOUD_OBJECT_PATH="rocksdb/fillrandom" # The path of the database stored in the bucket

hope my solution would help you😊