Log parser crash when there are few CF #25
Description
ubuntu@zIP-9-151-1-21:~/log-parser$ git branch -av
* main d600b0e Discard (not assert / fail) when compaction stats lines are badly formed (#22)
remotes/origin/18-display-the-relative-path-of-the-csv-s-in-the-json-rather-than-absolute-paths 80a8445 Create validation.yml (#3)
remotes/origin/19-assertionerror-dbwidestatsmngris_start_line 72b28a6 Support non-contiguous 'DUMPING STATS' and 'DB Stats' entries
remotes/origin/21-indexerror-parsing-a-log-file-with-cropped-compaction-stats-header-line 766da36 Discard (not assert / fail) when compaction stats lines are badly formed
remotes/origin/24-assertion-error-block-cache-parsing b560e96 Discard (rather than assert) badly formed Block cache stats lines (#24)
remotes/origin/HEAD -> origin/main
remotes/origin/main d600b0e Discard (not assert / fail) when compaction stats lines are badly formed (#22)
ubuntu@zIP-9-151-1-21:~/log-parser$ python3 log_parser.py -l /data/LOG.old.1693224943334021
Traceback (most recent call last):
File "log_parser.py", line 352, in <module>
main()
File "log_parser.py", line 292, in main
parsed_log = parse_log(log_file_path)
File "log_parser.py", line 73, in parse_log
return ParsedLog(log_file_path, log_lines,
File "/home/ubuntu/log-parser/log_file.py", line 184, in __init__
self.parse_rest_of_log()
File "/home/ubuntu/log-parser/log_file.py", line 470, in parse_rest_of_log
if self.try_parse_as_stats_entries():
File "/home/ubuntu/log-parser/log_file.py", line 412, in try_parse_as_stats_entries
self.stats_mngr.try_adding_entries(self.log_entries,
File "/home/ubuntu/log-parser/stats_mngr.py", line 1139, in try_adding_entries
self.parse_next_db_stats_entry_lines(
File "/home/ubuntu/log-parser/stats_mngr.py", line 1059, in parse_next_db_stats_entry_lines
self.block_cache_stats_mngr.add_lines(time, cf_name,
File "/home/ubuntu/log-parser/stats_mngr.py", line 864, in add_lines
self.parse_global_entry_stats_line(time, cache_id, db_stats_lines[1])
File "/home/ubuntu/log-parser/stats_mngr.py", line 887, in parse_global_entry_stats_line
roles, roles_stats = BlockCacheStatsMngr.parse_entry_stats_line(
File "/home/ubuntu/log-parser/stats_mngr.py", line 939, in parse_entry_stats_line
assert False, str(ParsingAssertion(
AssertionError: Error Parsing block cache stats line. roles:['FilterBlock', 'IndexBlock', 'WriteBuffer', 'FilterConstruction', 'BlockBasedTableReader', 'FileMetadata'], roles_stats:['4126,2.71 GB,67.7071%', '4144,911.92 MB,22.2637%', '993,248.25 MB,6.06079%', '14,3.50 MB,0.0854492%', '12,3.00 MB,0.0732422%'][File:? (line#:?)]
FilterBlock(4126,2.71 GB,67.7071%) IndexBlock(4144,911.92 MB,22.2637%) WriteBuffer(993,248.25 MB,6.06079%) FilterConstruction(14,3.50 MB,0.0854492%) BlockBasedTableReader(12,3.00 MB,0.0732422%) FileMetadata(12,3.00 MB,0.07324
Metadata
Metadata
Assignees
Labels
No labels