Clickhouse: Import compressed JSON fast
As part of the MVG Observatory Project I collect departures of the Munich public transport. These departures are stored on the filesystem in the following structure: 20240615/ ├── de:09162:1 │ ├── 1718409659_body.json │ ├── 1718409659_meta.json Top level is the date and the subfolder includes multiple stations (with their id as the folder name). Each station includes lots of single files. Every day the top level folder gets archived into a zstd compressed file....