2021-08-20 13:25:59 Hail: INFO: Running Hail version 0.2.74-0c3a74d12093 2021-08-20 13:25:59 SparkContext: WARN: Using an existing SparkContext; some configuration may not take effect. 2021-08-20 13:26:11 root: INFO: RegionPool: initialized for thread 28: Thread-5 2021-08-20 13:26:11 MemoryStore: INFO: Block broadcast_0 stored as values in memory (estimated size 2.3 MiB, free 26.5 GiB) 2021-08-20 13:26:12 MemoryStore: INFO: Block broadcast_0_piece0 stored as bytes in memory (estimated size 123.7 KiB, free 26.5 GiB) 2021-08-20 13:26:12 BlockManagerInfo: INFO: Added broadcast_0_piece0 in memory on spark-master:32903 (size: 123.7 KiB, free: 26.5 GiB) 2021-08-20 13:26:12 SparkContext: INFO: Created broadcast 0 from broadcast at SparkBackend.scala:278 2021-08-20 13:26:12 root: INFO: RegionPool: FREE: 64.0K allocated (64.0K blocks / 0 chunks), regions.size = 1, 0 current java objects, thread 28: Thread-5 2021-08-20 13:26:12 root: INFO: timing SparkBackend.parse_matrix_ir total 879.200ms self 879.200ms children 0.000ms %children 0.00% 2021-08-20 13:26:16 root: INFO: RegionPool: initialized for thread 28: Thread-5 2021-08-20 13:26:16 root: INFO: vep config json: JObject(List((name,JString(VEP)), (config,JString(/all_ssd/resources/vep_config_sv.json)), (csq,JBool(true)), (blockSize,JInt(1000)), (tolerateParseError,JBool(false)))) 2021-08-20 13:26:16 root: INFO: RegionPool: FREE: 64.0K allocated (64.0K blocks / 0 chunks), regions.size = 1, 0 current java objects, thread 28: Thread-5 2021-08-20 13:26:16 root: INFO: timing SparkBackend.parse_table_ir total 190.212ms self 190.212ms children 0.000ms %children 0.00% 2021-08-20 13:26:16 root: INFO: RegionPool: initialized for thread 28: Thread-5 2021-08-20 13:26:16 root: INFO: optimize optimize: legacyRelationalLowerer, initial IR: before: IR size 9: (TableToTableApply "{\"config\":\"/all_ssd/resources/vep_config_sv.json\",\"csq\":true,\"blockSize\":1000,\"tolerateParseError\":false,\"name\":\"VEP\"}" (TableDistinct (MatrixRowsTable (MatrixMapRows (MatrixRead None False False (MatrixNativeReader file:///home/a7420174/Downloads/test_sv_del.mt )) (InsertFields (SelectFields (locus alleles) (SelectFields (locus alleles rsid qual filters info) (Ref va))) None))))) 2021-08-20 13:26:16 root: INFO: optimize optimize: legacyRelationalLowerer, initial IR: after: IR size 4: (TableToTableApply "{\"config\":\"/all_ssd/resources/vep_config_sv.json\",\"csq\":true,\"blockSize\":1000,\"tolerateParseError\":false,\"name\":\"VEP\"}" (TableDistinct (MatrixRowsTable (MatrixRead Matrix{global:Struct{},col_key:[s],col:Struct{s:String},row_key:[[locus,alleles]],row:Struct{locus:Locus(GRCh38),alleles:Array[String]},entry:Struct{}} True False (MatrixNativeReader file:///home/a7420174/Downloads/test_sv_del.mt ))))) 2021-08-20 13:26:16 root: INFO: optimize optimize: legacyRelationalLowerer, after LowerMatrixToTable: before: IR size 17: (TableToTableApply "{\"config\":\"/all_ssd/resources/vep_config_sv.json\",\"csq\":true,\"blockSize\":1000,\"tolerateParseError\":false,\"name\":\"VEP\"}" (TableDistinct (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh38),alleles:Array[String]}} False (TableNativeReader file:///home/a7420174/Downloads/test_sv_del.mt/rows )) (InsertFields (Ref global) None (__cols (MakeArray Array[Struct{s:String}])))) (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (MakeArray Array[Struct{}])))) (SelectFields () (Ref global))) (SelectFields (locus alleles) (Ref row))))) 2021-08-20 13:26:17 root: INFO: Prune: InsertFields: eliminating field '__cols' 2021-08-20 13:26:17 root: INFO: Prune: InsertFields: eliminating field 'the entries! [877f12a8827e18f61222c6c8c5fb04a8]' 2021-08-20 13:26:17 root: INFO: optimize optimize: legacyRelationalLowerer, after LowerMatrixToTable: after: IR size 3: (TableToTableApply "{\"config\":\"/all_ssd/resources/vep_config_sv.json\",\"csq\":true,\"blockSize\":1000,\"tolerateParseError\":false,\"name\":\"VEP\"}" (TableDistinct (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh38),alleles:Array[String]}} False (TableNativeReader file:///home/a7420174/Downloads/test_sv_del.mt/rows )))) 2021-08-20 13:26:17 root: INFO: optimize optimize: legacyRelationalLowerer, after LegacyInterpretNonCompilable: before: IR size 3: (TableToTableApply "{\"config\":\"/all_ssd/resources/vep_config_sv.json\",\"csq\":true,\"blockSize\":1000,\"tolerateParseError\":false,\"name\":\"VEP\"}" (TableDistinct (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh38),alleles:Array[String]}} False (TableNativeReader file:///home/a7420174/Downloads/test_sv_del.mt/rows )))) 2021-08-20 13:26:17 root: INFO: optimize optimize: legacyRelationalLowerer, after LegacyInterpretNonCompilable: after: IR size 3: (TableToTableApply "{\"config\":\"/all_ssd/resources/vep_config_sv.json\",\"csq\":true,\"blockSize\":1000,\"tolerateParseError\":false,\"name\":\"VEP\"}" (TableDistinct (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh38),alleles:Array[String]}} False (TableNativeReader file:///home/a7420174/Downloads/test_sv_del.mt/rows )))) 2021-08-20 13:26:17 root: INFO: decoder cache miss (0 hits, 1 misses, 0.000 2021-08-20 13:26:17 root: INFO: instruction count: 3: __C1FSContainer. 2021-08-20 13:26:17 root: INFO: instruction count: 3: __C1FSContainer. 2021-08-20 13:26:17 root: INFO: instruction count: 3: __C2etypeDecode. 2021-08-20 13:26:17 root: INFO: instruction count: 5: __C2etypeDecode.apply 2021-08-20 13:26:17 root: INFO: instruction count: 16: __C2etypeDecode.__m3DECODE_r_struct_of_END_TO_o_struct_of_END 2021-08-20 13:26:17 root: INFO: decoder cache miss (0 hits, 2 misses, 0.000 2021-08-20 13:26:17 root: INFO: instruction count: 3: __C4FSContainer. 2021-08-20 13:26:17 root: INFO: instruction count: 3: __C4FSContainer. 2021-08-20 13:26:17 root: INFO: instruction count: 3: __C5etypeDecode. 2021-08-20 13:26:17 root: INFO: instruction count: 5: __C5etypeDecode.apply 2021-08-20 13:26:17 root: INFO: instruction count: 86: __C5etypeDecode.__m6DECODE_r_struct_of_r_struct_of_r_binaryANDr_int32ENDANDr_array_of_r_binaryANDo_binaryANDo_float64ANDo_array_of_r_binaryANDr_struct_of_r_boolANDo_binaryANDo_array_of_o_int32ANDo_int32ANDo_array_of_o_int32ANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_binaryANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_int32ANDo_int32ANDo_int32ENDEND_TO_o_struct_of_r_locusANDr_array_of_r_stringEND 2021-08-20 13:26:17 root: INFO: instruction count: 22: __C5etypeDecode.__m7INPLACE_DECODE_r_struct_of_r_binaryANDr_int32END_TO_r_locus 2021-08-20 13:26:17 root: INFO: instruction count: 33: __C5etypeDecode.__m8INPLACE_DECODE_r_binary_TO_r_string 2021-08-20 13:26:17 root: INFO: instruction count: 10: __C5etypeDecode.__m9INPLACE_DECODE_r_int32_TO_r_int32 2021-08-20 13:26:17 root: INFO: instruction count: 64: __C5etypeDecode.__m10INPLACE_DECODE_r_array_of_r_binary_TO_r_array_of_r_string 2021-08-20 13:26:17 root: INFO: instruction count: 7: __C5etypeDecode.__m11SKIP_o_binary 2021-08-20 13:26:17 root: INFO: instruction count: 3: __C5etypeDecode.__m12SKIP_o_float64 2021-08-20 13:26:17 root: INFO: instruction count: 22: __C5etypeDecode.__m13SKIP_o_array_of_r_binary 2021-08-20 13:26:17 root: INFO: instruction count: 7: __C5etypeDecode.__m14SKIP_r_binary 2021-08-20 13:26:17 root: INFO: instruction count: 264: __C5etypeDecode.__m15SKIP_r_struct_of_r_boolANDo_binaryANDo_array_of_o_int32ANDo_int32ANDo_array_of_o_int32ANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_binaryANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_int32ANDo_int32ANDo_int32END 2021-08-20 13:26:17 root: INFO: instruction count: 3: __C5etypeDecode.__m16SKIP_r_bool 2021-08-20 13:26:17 root: INFO: instruction count: 51: __C5etypeDecode.__m17SKIP_o_array_of_o_int32 2021-08-20 13:26:17 root: INFO: instruction count: 3: __C5etypeDecode.__m18SKIP_o_int32 2021-08-20 13:26:17 root: INFO: instruction count: 51: __C5etypeDecode.__m19SKIP_o_array_of_o_binary 2021-08-20 13:26:17 MemoryStore: INFO: Block broadcast_1 stored as values in memory (estimated size 375.7 KiB, free 26.5 GiB) 2021-08-20 13:26:17 MemoryStore: INFO: Block broadcast_1_piece0 stored as bytes in memory (estimated size 28.2 KiB, free 26.5 GiB) 2021-08-20 13:26:17 BlockManagerInfo: INFO: Added broadcast_1_piece0 in memory on spark-master:32903 (size: 28.2 KiB, free: 26.5 GiB) 2021-08-20 13:26:17 SparkContext: INFO: Created broadcast 1 from broadcast at SparkBackend.scala:278 2021-08-20 13:26:19 root: INFO: encoder cache miss (0 hits, 1 misses, 0.000) 2021-08-20 13:26:19 root: INFO: instruction count: 3: __C20FSContainer. 2021-08-20 13:26:19 root: INFO: instruction count: 3: __C20FSContainer. 2021-08-20 13:26:19 root: INFO: instruction count: 3: __C21etypeEncode. 2021-08-20 13:26:19 root: INFO: instruction count: 5: __C21etypeEncode.apply 2021-08-20 13:26:19 root: INFO: instruction count: 65: __C21etypeEncode.__m22ENCODE_o_struct_of_r_locusANDr_array_of_r_stringANDo_array_of_o_stringANDo_struct_of_o_int32ANDo_int32ENDEND_TO_r_struct_of_r_struct_of_r_binaryANDr_int32ENDANDr_array_of_r_binaryANDo_array_of_o_binaryANDo_struct_of_o_int32ANDo_int32ENDEND 2021-08-20 13:26:19 root: INFO: instruction count: 39: __C21etypeEncode.__m23ENCODE_o_locus_TO_r_struct_of_r_binaryANDr_int32END 2021-08-20 13:26:19 root: INFO: instruction count: 18: __C21etypeEncode.__m24ENCODE_o_string_TO_r_binary 2021-08-20 13:26:19 root: INFO: instruction count: 6: __C21etypeEncode.__m25ENCODE_o_int32_TO_r_int32 2021-08-20 13:26:19 root: INFO: instruction count: 49: __C21etypeEncode.__m26ENCODE_o_array_of_r_string_TO_r_array_of_r_binary 2021-08-20 13:26:19 root: INFO: instruction count: 125: __C21etypeEncode.__m27ENCODE_o_array_of_o_string_TO_o_array_of_o_binary 2021-08-20 13:26:19 root: INFO: instruction count: 18: __C21etypeEncode.__m28ENCODE_o_string_TO_o_binary 2021-08-20 13:26:19 root: INFO: instruction count: 53: __C21etypeEncode.__m29ENCODE_o_struct_of_o_int32ANDo_int32END_TO_o_struct_of_o_int32ANDo_int32END 2021-08-20 13:26:19 root: INFO: instruction count: 6: __C21etypeEncode.__m30ENCODE_o_int32_TO_o_int32 2021-08-20 13:26:19 root: INFO: decoder cache miss (0 hits, 3 misses, 0.000 2021-08-20 13:26:19 root: INFO: instruction count: 3: __C31FSContainer. 2021-08-20 13:26:19 root: INFO: instruction count: 3: __C31FSContainer. 2021-08-20 13:26:19 root: INFO: instruction count: 3: __C32etypeDecode. 2021-08-20 13:26:19 root: INFO: instruction count: 5: __C32etypeDecode.apply 2021-08-20 13:26:19 root: INFO: instruction count: 93: __C32etypeDecode.__m33DECODE_r_struct_of_r_struct_of_r_binaryANDr_int32ENDANDr_array_of_r_binaryANDo_array_of_o_binaryANDo_struct_of_o_int32ANDo_int32ENDEND_TO_o_struct_of_r_locusANDr_array_of_r_stringANDo_array_of_o_stringANDo_struct_of_o_int32ANDo_int32ENDEND 2021-08-20 13:26:19 root: INFO: instruction count: 22: __C32etypeDecode.__m34INPLACE_DECODE_r_struct_of_r_binaryANDr_int32END_TO_r_locus 2021-08-20 13:26:19 root: INFO: instruction count: 33: __C32etypeDecode.__m35INPLACE_DECODE_r_binary_TO_r_string 2021-08-20 13:26:19 root: INFO: instruction count: 10: __C32etypeDecode.__m36INPLACE_DECODE_r_int32_TO_r_int32 2021-08-20 13:26:19 root: INFO: instruction count: 64: __C32etypeDecode.__m37INPLACE_DECODE_r_array_of_r_binary_TO_r_array_of_r_string 2021-08-20 13:26:19 root: INFO: instruction count: 120: __C32etypeDecode.__m38INPLACE_DECODE_o_array_of_o_binary_TO_o_array_of_o_string 2021-08-20 13:26:19 root: INFO: instruction count: 33: __C32etypeDecode.__m39INPLACE_DECODE_o_binary_TO_o_string 2021-08-20 13:26:19 root: INFO: instruction count: 71: __C32etypeDecode.__m40INPLACE_DECODE_o_struct_of_o_int32ANDo_int32END_TO_o_struct_of_o_int32ANDo_int32END 2021-08-20 13:26:19 root: INFO: instruction count: 10: __C32etypeDecode.__m41INPLACE_DECODE_o_int32_TO_o_int32 2021-08-20 13:26:19 root: INFO: encoder cache miss (0 hits, 2 misses, 0.000) 2021-08-20 13:26:19 root: INFO: instruction count: 3: __C42FSContainer. 2021-08-20 13:26:19 root: INFO: instruction count: 3: __C42FSContainer. 2021-08-20 13:26:19 root: INFO: instruction count: 3: __C43etypeEncode. 2021-08-20 13:26:19 root: INFO: instruction count: 5: __C43etypeEncode.apply 2021-08-20 13:26:19 root: INFO: instruction count: 13: __C43etypeEncode.__m44ENCODE_o_struct_of_r_stringEND_TO_r_struct_of_r_binaryEND 2021-08-20 13:26:19 root: INFO: instruction count: 18: __C43etypeEncode.__m45ENCODE_o_string_TO_r_binary 2021-08-20 13:26:19 root: INFO: RegionPool: FREE: 128.0K allocated (128.0K blocks / 0 chunks), regions.size = 2, 0 current java objects, thread 28: Thread-5 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable total 2.639s self 2.365s children 274.407ms %children 10.40% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR total 230.295ms self 0.822ms children 229.473ms %children 99.64% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/Verify total 2.482ms self 2.482ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation total 226.967ms self 143.814ms children 83.153ms %children 36.64% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize total 83.153ms self 1.650ms children 81.502ms %children 98.02% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 5.521ms self 5.521ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 8.404ms self 8.404ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 20.143ms self 20.143ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 19.232ms self 19.232ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 1.587ms self 1.587ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 23.851ms self 23.851ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.176ms self 0.176ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.111ms self 0.111ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.236ms self 0.236ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.343ms self 0.343ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.032ms self 0.032ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.669ms self 0.669ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.096ms self 0.096ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.019ms self 0.019ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.058ms self 0.058ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.311ms self 0.311ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.025ms self 0.025ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.690ms self 0.690ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, initial IR/Verify total 0.024ms self 0.024ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/LowerMatrixToTable total 16.601ms self 0.015ms children 16.587ms %children 99.91% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/LowerMatrixToTable/Verify total 0.014ms self 0.014ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/LowerMatrixToTable/LoweringTransformation total 16.436ms self 16.436ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/LowerMatrixToTable/Verify total 0.136ms self 0.136ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable total 23.228ms self 0.015ms children 23.213ms %children 99.94% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/Verify total 0.037ms self 0.037ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation total 23.160ms self 5.917ms children 17.243ms %children 74.45% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize total 17.243ms self 0.116ms children 17.128ms %children 99.33% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/FoldConstants total 2.169ms self 2.169ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.274ms self 0.274ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/Simplify total 0.674ms self 0.674ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardLets total 2.709ms self 2.709ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.156ms self 0.156ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/PruneDeadFields total 8.451ms self 8.451ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/FoldConstants total 0.407ms self 0.407ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.228ms self 0.228ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/Simplify total 0.426ms self 0.426ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardLets total 0.276ms self 0.276ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.020ms self 0.020ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/PruneDeadFields total 0.481ms self 0.481ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/FoldConstants total 0.078ms self 0.078ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.032ms self 0.032ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/Simplify total 0.041ms self 0.041ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardLets total 0.263ms self 0.263ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.019ms self 0.019ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/PruneDeadFields total 0.428ms self 0.428ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LowerMatrixToTable/Verify total 0.016ms self 0.016ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/InterpretNonCompilable total 1.435ms self 0.011ms children 1.423ms %children 99.21% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/InterpretNonCompilable/Verify total 0.012ms self 0.012ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/InterpretNonCompilable/LoweringTransformation total 1.382ms self 1.382ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/InterpretNonCompilable/Verify total 0.030ms self 0.030ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LegacyInterpretNonCompilable total 2.848ms self 0.011ms children 2.837ms %children 99.61% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LegacyInterpretNonCompilable/Verify total 0.007ms self 0.007ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LegacyInterpretNonCompilable/LoweringTransformation total 2.817ms self 1.819ms children 0.998ms %children 35.43% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LegacyInterpretNonCompilable/LoweringTransformation/Optimize total 0.998ms self 0.061ms children 0.938ms %children 93.92% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LegacyInterpretNonCompilable/LoweringTransformation/Optimize/FoldConstants total 0.069ms self 0.069ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LegacyInterpretNonCompilable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.045ms self 0.045ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LegacyInterpretNonCompilable/LoweringTransformation/Optimize/Simplify total 0.044ms self 0.044ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LegacyInterpretNonCompilable/LoweringTransformation/Optimize/ForwardLets total 0.302ms self 0.302ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LegacyInterpretNonCompilable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.021ms self 0.021ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LegacyInterpretNonCompilable/LoweringTransformation/Optimize/PruneDeadFields total 0.456ms self 0.456ms children 0.000ms %children 0.00% 2021-08-20 13:26:19 root: INFO: timing SparkBackend.pyPersistTable/optimize: legacyRelationalLowerer, after LegacyInterpretNonCompilable/Verify total 0.012ms self 0.012ms children 0.000ms %children 0.00% 2021-08-20 13:26:26 root: INFO: RegionPool: initialized for thread 28: Thread-5 2021-08-20 13:26:26 root: INFO: RegionPool: FREE: 64.0K allocated (64.0K blocks / 0 chunks), regions.size = 1, 0 current java objects, thread 28: Thread-5 2021-08-20 13:26:26 root: INFO: timing SparkBackend.parse_value_ir total 178.009ms self 178.009ms children 0.000ms %children 0.00% 2021-08-20 13:26:26 root: INFO: RegionPool: initialized for thread 28: Thread-5 2021-08-20 13:26:26 root: INFO: starting execution of query hail_query_1 of initial size 172 2021-08-20 13:26:26 root: INFO: optimize optimize: relationalLowerer, initial IR: before: IR size 172: (GetField rows (TableCollect (TableHead 11 (TableMapRows (TableMapRows (TableOrderBy (Alocus Aalleles) (MatrixRowsTable (MatrixMapRows (MatrixAnnotateRowsTable "__uid_3" False (MatrixMapGlobals (MatrixRead None False False (MatrixNativeReader file:///home/a7420174/Downloads/test_sv_del.mt )) (InsertFields (SelectFields () (Ref global)) None (vep_csq_header (GetField vep_csq_header (TableGetGlobals (TableLiteral)))))) (TableLiteral)) (Let __uid_4 (GetField __uid_3 (Ref va)) (InsertFields (SelectFields (locus alleles rsid qual filters info) (Ref va)) None (vep (GetField vep (Ref __uid_4))) (vep_proc_id (GetField vep_proc_id (Ref __uid_4)))))))) (Let __uid_6 (GetField vep_proc_id (Ref row)) (Let __uid_5 (GetField info (Ref row)) (InsertFields (SelectFields () (SelectFields (locus alleles rsid qual filters info vep vep_proc_id) (Ref row))) None (locus (GetField locus (Ref row))) (alleles (GetField alleles (Ref row))) (rsid (GetField rsid (Ref row))) (qual (GetField qual (Ref row))) (filters (GetField filters (Ref row))) (`info.IMPRECISE` (GetField IMPRECISE (Ref __uid_5))) (`info.SVTYPE` (GetField SVTYPE (Ref __uid_5))) (`info.SVLEN` (GetField SVLEN (Ref __uid_5))) (`info.END` (GetField END (Ref __uid_5))) (`info.CIPOS` (GetField CIPOS (Ref __uid_5))) (`info.CIEND` (GetField CIEND (Ref __uid_5))) (`info.CIGAR` (GetField CIGAR (Ref __uid_5))) (`info.MATEID` (GetField MATEID (Ref __uid_5))) (`info.EVENT` (GetField EVENT (Ref __uid_5))) (`info.HOMLEN` (GetField HOMLEN (Ref __uid_5))) (`info.HOMSEQ` (GetField HOMSEQ (Ref __uid_5))) (`info.SVINSLEN` (GetField SVINSLEN (Ref __uid_5))) (`info.SVINSSEQ` (GetField SVINSSEQ (Ref __uid_5))) (`info.LEFT_SVINSSEQ` (GetField LEFT_SVINSSEQ (Ref __uid_5))) (`info.RIGHT_SVINSSEQ` (GetField RIGHT_SVINSSEQ (Ref __uid_5))) (`info.BND_DEPTH` (GetField BND_DEPTH (Ref __uid_5))) (`info.MATE_BND_DEPTH` (GetField MATE_BND_DEPTH (Ref __uid_5))) (`info.JUNCTION_QUAL` (GetField JUNCTION_QUAL (Ref __uid_5))) (vep (GetField vep (Ref row))) (`vep_proc_id.part_idx` (GetField part_idx (Ref __uid_6))) (`vep_proc_id.block_idx` (GetField block_idx (Ref __uid_6))))))) (InsertFields (SelectFields () (SelectFields (locus alleles rsid qual filters `info.IMPRECISE` `info.SVTYPE` `info.SVLEN` `info.END` `info.CIPOS` `info.CIEND` `info.CIGAR` `info.MATEID` `info.EVENT` `info.HOMLEN` `info.HOMSEQ` `info.SVINSLEN` `info.SVINSSEQ` `info.LEFT_SVINSSEQ` `info.RIGHT_SVINSSEQ` `info.BND_DEPTH` `info.MATE_BND_DEPTH` `info.JUNCTION_QUAL` vep `vep_proc_id.part_idx` `vep_proc_id.block_idx`) (Ref row))) None (locus (ApplySpecial 1 showStr () String (GetField locus (Ref row)))) (alleles (ApplySpecial 2 showStr () String (GetField alleles (Ref row)))) (rsid (ApplySpecial 3 showStr () String (GetField rsid (Ref row)))) (qual (ApplySpecial 4 showStr () String (GetField qual (Ref row)))) (filters (ApplySpecial 5 showStr () String (GetField filters (Ref row)))) (`info.IMPRECISE` (ApplySpecial 6 showStr () String (GetField `info.IMPRECISE` (Ref row)))) (`info.SVTYPE` (ApplySpecial 7 showStr () String (GetField `info.SVTYPE` (Ref row)))) (`info.SVLEN` (ApplySpecial 8 showStr () String (GetField `info.SVLEN` (Ref row)))) (`info.END` (ApplySpecial 9 showStr () String (GetField `info.END` (Ref row)))) (`info.CIPOS` (ApplySpecial 10 showStr () String (GetField `info.CIPOS` (Ref row)))) (`info.CIEND` (ApplySpecial 11 showStr () String (GetField `info.CIEND` (Ref row)))) (`info.CIGAR` (ApplySpecial 12 showStr () String (GetField `info.CIGAR` (Ref row)))) (`info.MATEID` (ApplySpecial 13 showStr () String (GetField `info.MATEID` (Ref row)))) (`info.EVENT` (ApplySpecial 14 showStr () String (GetField `info.EVENT` (Ref row)))) (`info.HOMLEN` (ApplySpecial 15 showStr () String (GetField `info.HOMLEN` (Ref row)))) (`info.HOMSEQ` (ApplySpecial 16 showStr () String (GetField `info.HOMSEQ` (Ref row)))) (`info.SVINSLEN` (ApplySpecial 17 showStr () String (GetField `info.SVINSLEN` (Ref row)))) (`info.SVINSSEQ` (ApplySpecial 18 showStr () String (GetField `info.SVINSSEQ` (Ref row)))) (`info.LEFT_SVINSSEQ` (ApplySpecial 19 showStr () String (GetField `info.LEFT_SVINSSEQ` (Ref row)))) (`info.RIGHT_SVINSSEQ` (ApplySpecial 20 showStr () String (GetField `info.RIGHT_SVINSSEQ` (Ref row)))) (`info.BND_DEPTH` (ApplySpecial 21 showStr () String (GetField `info.BND_DEPTH` (Ref row)))) (`info.MATE_BND_DEPTH` (ApplySpecial 22 showStr () String (GetField `info.MATE_BND_DEPTH` (Ref row)))) (`info.JUNCTION_QUAL` (ApplySpecial 23 showStr () String (GetField `info.JUNCTION_QUAL` (Ref row)))) (vep (ApplySpecial 24 showStr () String (GetField vep (Ref row)))) (`vep_proc_id.part_idx` (ApplySpecial 25 showStr () String (GetField `vep_proc_id.part_idx` (Ref row)))) (`vep_proc_id.block_idx` (ApplySpecial 26 showStr () String (GetField `vep_proc_id.block_idx` (Ref row))))))))) 2021-08-20 13:26:26 root: INFO: Prune: InsertFields: eliminating field 'vep_csq_header' 2021-08-20 13:26:26 root: INFO: optimize optimize: relationalLowerer, initial IR: after: IR size 110: (GetField rows (TableCollect (TableMapRows (TableHead 11 (TableOrderBy (Alocus Aalleles) (MatrixRowsTable (MatrixMapRows (MatrixAnnotateRowsTable "__uid_3" False (MatrixRead Matrix{global:Struct{},col_key:[s],col:Struct{s:String},row_key:[[locus,alleles]],row:Struct{locus:Locus(GRCh38),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{IMPRECISE:Boolean,SVTYPE:String,SVLEN:Array[Int32],END:Int32,CIPOS:Array[Int32],CIEND:Array[Int32],CIGAR:Array[String],MATEID:Array[String],EVENT:String,HOMLEN:Array[Int32],HOMSEQ:Array[String],SVINSLEN:Array[Int32],SVINSSEQ:Array[String],LEFT_SVINSSEQ:Array[String],RIGHT_SVINSSEQ:Array[String],BND_DEPTH:Int32,MATE_BND_DEPTH:Int32,JUNCTION_QUAL:Int32}},entry:Struct{}} False False (MatrixNativeReader file:///home/a7420174/Downloads/test_sv_del.mt )) (TableLiteral)) (Let __iruid_69 (GetField __uid_3 (Ref va)) (InsertFields (SelectFields (locus alleles rsid qual filters info) (Ref va)) None (vep (GetField vep (Ref __iruid_69))) (vep_proc_id (GetField vep_proc_id (Ref __iruid_69))))))))) (Let __iruid_70 (GetField vep_proc_id (Ref row)) (Let __iruid_71 (GetField info (Ref row)) (Let __iruid_72 (SelectFields (locus alleles rsid qual filters vep) (Ref row)) (InsertFields (SelectFields () (Ref __iruid_72)) None (locus (ApplySpecial 1 showStr () String (GetField locus (Ref __iruid_72)))) (alleles (ApplySpecial 2 showStr () String (GetField alleles (Ref __iruid_72)))) (rsid (ApplySpecial 3 showStr () String (GetField rsid (Ref __iruid_72)))) (qual (ApplySpecial 4 showStr () String (GetField qual (Ref __iruid_72)))) (filters (ApplySpecial 5 showStr () String (GetField filters (Ref __iruid_72)))) (`info.IMPRECISE` (ApplySpecial 6 showStr () String (GetField IMPRECISE (Ref __iruid_71)))) (`info.SVTYPE` (ApplySpecial 7 showStr () String (GetField SVTYPE (Ref __iruid_71)))) (`info.SVLEN` (ApplySpecial 8 showStr () String (GetField SVLEN (Ref __iruid_71)))) (`info.END` (ApplySpecial 9 showStr () String (GetField END (Ref __iruid_71)))) (`info.CIPOS` (ApplySpecial 10 showStr () String (GetField CIPOS (Ref __iruid_71)))) (`info.CIEND` (ApplySpecial 11 showStr () String (GetField CIEND (Ref __iruid_71)))) (`info.CIGAR` (ApplySpecial 12 showStr () String (GetField CIGAR (Ref __iruid_71)))) (`info.MATEID` (ApplySpecial 13 showStr () String (GetField MATEID (Ref __iruid_71)))) (`info.EVENT` (ApplySpecial 14 showStr () String (GetField EVENT (Ref __iruid_71)))) (`info.HOMLEN` (ApplySpecial 15 showStr () String (GetField HOMLEN (Ref __iruid_71)))) (`info.HOMSEQ` (ApplySpecial 16 showStr () String (GetField HOMSEQ (Ref __iruid_71)))) (`info.SVINSLEN` (ApplySpecial 17 showStr () String (GetField SVINSLEN (Ref __iruid_71)))) (`info.SVINSSEQ` (ApplySpecial 18 showStr () String (GetField SVINSSEQ (Ref __iruid_71)))) (`info.LEFT_SVINSSEQ` (ApplySpecial 19 showStr () String (GetField LEFT_SVINSSEQ (Ref __iruid_71)))) (`info.RIGHT_SVINSSEQ` (ApplySpecial 20 showStr () String (GetField RIGHT_SVINSSEQ (Ref __iruid_71)))) (`info.BND_DEPTH` (ApplySpecial 21 showStr () String (GetField BND_DEPTH (Ref __iruid_71)))) (`info.MATE_BND_DEPTH` (ApplySpecial 22 showStr () String (GetField MATE_BND_DEPTH (Ref __iruid_71)))) (`info.JUNCTION_QUAL` (ApplySpecial 23 showStr () String (GetField JUNCTION_QUAL (Ref __iruid_71)))) (vep (ApplySpecial 24 showStr () String (GetField vep (Ref __iruid_72)))) (`vep_proc_id.part_idx` (ApplySpecial 25 showStr () String (GetField part_idx (Ref __iruid_70)))) (`vep_proc_id.block_idx` (ApplySpecial 26 showStr () String (GetField block_idx (Ref __iruid_70))))))))))) 2021-08-20 13:26:27 root: INFO: optimize optimize: relationalLowerer, after LowerMatrixToTable: before: IR size 132: (GetField rows (TableCollect (TableMapRows (TableHead 11 (TableOrderBy (Alocus Aalleles) (TableMapRows (TableMapGlobals (TableMapRows (TableLeftJoinRightDistinct __uid_3 (TableMapGlobals (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh38),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{IMPRECISE:Boolean,SVTYPE:String,SVLEN:Array[Int32],END:Int32,CIPOS:Array[Int32],CIEND:Array[Int32],CIGAR:Array[String],MATEID:Array[String],EVENT:String,HOMLEN:Array[Int32],HOMSEQ:Array[String],SVINSLEN:Array[Int32],SVINSSEQ:Array[String],LEFT_SVINSSEQ:Array[String],RIGHT_SVINSSEQ:Array[String],BND_DEPTH:Int32,MATE_BND_DEPTH:Int32,JUNCTION_QUAL:Int32},`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{}]}} False (TableNativeZippedReader file:///home/a7420174/Downloads/test_sv_del.mt/rows file:///home/a7420174/Downloads/test_sv_del.mt/entries )) (InsertFields (Ref global) None (__cols (ToArray (ReadPartition Struct{s:String} "{\"name\":\"PartitionNativeReader\",\"spec\":{\"name\":\"TypedCodecSpec\",\"_eType\":\"+EBaseStruct{s:+EBinary}\",\"_vType\":\"Struct{s:String}\",\"_bufferSpec\":{\"name\":\"LEB128BufferSpec\",\"child\":{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"LZ4HCBlockBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}}}}}" (Str "file:///ho...")))))) (TableLiteral)) (Let n_cols (ArrayLen (GetField __cols (Ref global))) (InsertFields (Let __iruid_73 (MakeStruct) (Let __iruid_69 (GetField __uid_3 (SelectFields (locus alleles rsid qual filters info __uid_3) (Ref row))) (InsertFields (SelectFields (locus alleles rsid qual filters info) (SelectFields (locus alleles rsid qual filters info __uid_3) (Ref row))) None (vep (GetField vep (Ref __iruid_69))) (vep_proc_id (GetField vep_proc_id (Ref __iruid_69)))))) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)))))) (SelectFields () (Ref global))) (SelectFields (locus alleles rsid qual filters info vep vep_proc_id) (Ref row))))) (Let __iruid_70 (GetField vep_proc_id (Ref row)) (Let __iruid_71 (GetField info (Ref row)) (Let __iruid_72 (SelectFields (locus alleles rsid qual filters vep) (Ref row)) (InsertFields (SelectFields () (Ref __iruid_72)) None (locus (ApplySpecial 1 showStr () String (GetField locus (Ref __iruid_72)))) (alleles (ApplySpecial 2 showStr () String (GetField alleles (Ref __iruid_72)))) (rsid (ApplySpecial 3 showStr () String (GetField rsid (Ref __iruid_72)))) (qual (ApplySpecial 4 showStr () String (GetField qual (Ref __iruid_72)))) (filters (ApplySpecial 5 showStr () String (GetField filters (Ref __iruid_72)))) (`info.IMPRECISE` (ApplySpecial 6 showStr () String (GetField IMPRECISE (Ref __iruid_71)))) (`info.SVTYPE` (ApplySpecial 7 showStr () String (GetField SVTYPE (Ref __iruid_71)))) (`info.SVLEN` (ApplySpecial 8 showStr () String (GetField SVLEN (Ref __iruid_71)))) (`info.END` (ApplySpecial 9 showStr () String (GetField END (Ref __iruid_71)))) (`info.CIPOS` (ApplySpecial 10 showStr () String (GetField CIPOS (Ref __iruid_71)))) (`info.CIEND` (ApplySpecial 11 showStr () String (GetField CIEND (Ref __iruid_71)))) (`info.CIGAR` (ApplySpecial 12 showStr () String (GetField CIGAR (Ref __iruid_71)))) (`info.MATEID` (ApplySpecial 13 showStr () String (GetField MATEID (Ref __iruid_71)))) (`info.EVENT` (ApplySpecial 14 showStr () String (GetField EVENT (Ref __iruid_71)))) (`info.HOMLEN` (ApplySpecial 15 showStr () String (GetField HOMLEN (Ref __iruid_71)))) (`info.HOMSEQ` (ApplySpecial 16 showStr () String (GetField HOMSEQ (Ref __iruid_71)))) (`info.SVINSLEN` (ApplySpecial 17 showStr () String (GetField SVINSLEN (Ref __iruid_71)))) (`info.SVINSSEQ` (ApplySpecial 18 showStr () String (GetField SVINSSEQ (Ref __iruid_71)))) (`info.LEFT_SVINSSEQ` (ApplySpecial 19 showStr () String (GetField LEFT_SVINSSEQ (Ref __iruid_71)))) (`info.RIGHT_SVINSSEQ` (ApplySpecial 20 showStr () String (GetField RIGHT_SVINSSEQ (Ref __iruid_71)))) (`info.BND_DEPTH` (ApplySpecial 21 showStr () String (GetField BND_DEPTH (Ref __iruid_71)))) (`info.MATE_BND_DEPTH` (ApplySpecial 22 showStr () String (GetField MATE_BND_DEPTH (Ref __iruid_71)))) (`info.JUNCTION_QUAL` (ApplySpecial 23 showStr () String (GetField JUNCTION_QUAL (Ref __iruid_71)))) (vep (ApplySpecial 24 showStr () String (GetField vep (Ref __iruid_72)))) (`vep_proc_id.part_idx` (ApplySpecial 25 showStr () String (GetField part_idx (Ref __iruid_70)))) (`vep_proc_id.block_idx` (ApplySpecial 26 showStr () String (GetField block_idx (Ref __iruid_70))))))))))) 2021-08-20 13:26:27 root: INFO: Prune: InsertFields: eliminating field '__cols' 2021-08-20 13:26:27 root: INFO: Prune: InsertFields: eliminating field 'the entries! [877f12a8827e18f61222c6c8c5fb04a8]' 2021-08-20 13:26:27 root: INFO: optimize optimize: relationalLowerer, after LowerMatrixToTable: after: IR size 109: (GetField rows (TableCollect (TableMapRows (TableHead 11 (TableOrderBy (Alocus Aalleles) (TableMapRows (TableLeftJoinRightDistinct __uid_3 (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh38),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{IMPRECISE:Boolean,SVTYPE:String,SVLEN:Array[Int32],END:Int32,CIPOS:Array[Int32],CIEND:Array[Int32],CIGAR:Array[String],MATEID:Array[String],EVENT:String,HOMLEN:Array[Int32],HOMSEQ:Array[String],SVINSLEN:Array[Int32],SVINSSEQ:Array[String],LEFT_SVINSSEQ:Array[String],RIGHT_SVINSSEQ:Array[String],BND_DEPTH:Int32,MATE_BND_DEPTH:Int32,JUNCTION_QUAL:Int32}}} False (TableNativeZippedReader file:///home/a7420174/Downloads/test_sv_del.mt/rows file:///home/a7420174/Downloads/test_sv_del.mt/entries )) (TableLiteral)) (Let __iruid_84 (GetField __uid_3 (Ref row)) (InsertFields (SelectFields (locus alleles rsid qual filters info) (Ref row)) None (vep (GetField vep (Ref __iruid_84))) (vep_proc_id (GetField vep_proc_id (Ref __iruid_84)))))))) (Let __iruid_85 (GetField vep_proc_id (Ref row)) (Let __iruid_86 (GetField info (Ref row)) (Let __iruid_87 (SelectFields (locus alleles rsid qual filters vep) (Ref row)) (InsertFields (SelectFields () (Ref __iruid_87)) None (locus (ApplySpecial 1 showStr () String (GetField locus (Ref __iruid_87)))) (alleles (ApplySpecial 2 showStr () String (GetField alleles (Ref __iruid_87)))) (rsid (ApplySpecial 3 showStr () String (GetField rsid (Ref __iruid_87)))) (qual (ApplySpecial 4 showStr () String (GetField qual (Ref __iruid_87)))) (filters (ApplySpecial 5 showStr () String (GetField filters (Ref __iruid_87)))) (`info.IMPRECISE` (ApplySpecial 6 showStr () String (GetField IMPRECISE (Ref __iruid_86)))) (`info.SVTYPE` (ApplySpecial 7 showStr () String (GetField SVTYPE (Ref __iruid_86)))) (`info.SVLEN` (ApplySpecial 8 showStr () String (GetField SVLEN (Ref __iruid_86)))) (`info.END` (ApplySpecial 9 showStr () String (GetField END (Ref __iruid_86)))) (`info.CIPOS` (ApplySpecial 10 showStr () String (GetField CIPOS (Ref __iruid_86)))) (`info.CIEND` (ApplySpecial 11 showStr () String (GetField CIEND (Ref __iruid_86)))) (`info.CIGAR` (ApplySpecial 12 showStr () String (GetField CIGAR (Ref __iruid_86)))) (`info.MATEID` (ApplySpecial 13 showStr () String (GetField MATEID (Ref __iruid_86)))) (`info.EVENT` (ApplySpecial 14 showStr () String (GetField EVENT (Ref __iruid_86)))) (`info.HOMLEN` (ApplySpecial 15 showStr () String (GetField HOMLEN (Ref __iruid_86)))) (`info.HOMSEQ` (ApplySpecial 16 showStr () String (GetField HOMSEQ (Ref __iruid_86)))) (`info.SVINSLEN` (ApplySpecial 17 showStr () String (GetField SVINSLEN (Ref __iruid_86)))) (`info.SVINSSEQ` (ApplySpecial 18 showStr () String (GetField SVINSSEQ (Ref __iruid_86)))) (`info.LEFT_SVINSSEQ` (ApplySpecial 19 showStr () String (GetField LEFT_SVINSSEQ (Ref __iruid_86)))) (`info.RIGHT_SVINSSEQ` (ApplySpecial 20 showStr () String (GetField RIGHT_SVINSSEQ (Ref __iruid_86)))) (`info.BND_DEPTH` (ApplySpecial 21 showStr () String (GetField BND_DEPTH (Ref __iruid_86)))) (`info.MATE_BND_DEPTH` (ApplySpecial 22 showStr () String (GetField MATE_BND_DEPTH (Ref __iruid_86)))) (`info.JUNCTION_QUAL` (ApplySpecial 23 showStr () String (GetField JUNCTION_QUAL (Ref __iruid_86)))) (vep (ApplySpecial 24 showStr () String (GetField vep (Ref __iruid_87)))) (`vep_proc_id.part_idx` (ApplySpecial 25 showStr () String (GetField part_idx (Ref __iruid_85)))) (`vep_proc_id.block_idx` (ApplySpecial 26 showStr () String (GetField block_idx (Ref __iruid_85))))))))))) 2021-08-20 13:26:27 root: INFO: interpreting non compilable node: TableCollect 2021-08-20 13:26:27 root: INFO: decoder cache hit 2021-08-20 13:26:27 root: INFO: decoder cache miss (1 hits, 4 misses, 0.200 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C46FSContainer. 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C46FSContainer. 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C47etypeDecode. 2021-08-20 13:26:27 root: INFO: instruction count: 5: __C47etypeDecode.apply 2021-08-20 13:26:27 root: INFO: instruction count: 125: __C47etypeDecode.__m48DECODE_r_struct_of_r_struct_of_r_binaryANDr_int32ENDANDr_array_of_r_binaryANDo_binaryANDo_float64ANDo_array_of_r_binaryANDr_struct_of_r_boolANDo_binaryANDo_array_of_o_int32ANDo_int32ANDo_array_of_o_int32ANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_binaryANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_int32ANDo_int32ANDo_int32ENDEND_TO_o_struct_of_r_locusANDr_array_of_r_stringANDo_stringANDo_float64ANDo_set_of_r_stringANDr_struct_of_r_boolANDo_stringANDo_array_of_o_int32ANDo_int32ANDo_array_of_o_int32ANDo_array_of_o_int32ANDo_array_of_o_stringANDo_array_of_o_stringANDo_stringANDo_array_of_o_int32ANDo_array_of_o_stringANDo_array_of_o_int32ANDo_array_of_o_stringANDo_array_of_o_stringANDo_array_of_o_stringANDo_int32ANDo_int32ANDo_int32ENDEND 2021-08-20 13:26:27 root: INFO: instruction count: 22: __C47etypeDecode.__m49INPLACE_DECODE_r_struct_of_r_binaryANDr_int32END_TO_r_locus 2021-08-20 13:26:27 root: INFO: instruction count: 33: __C47etypeDecode.__m50INPLACE_DECODE_r_binary_TO_r_string 2021-08-20 13:26:27 root: INFO: instruction count: 10: __C47etypeDecode.__m51INPLACE_DECODE_r_int32_TO_r_int32 2021-08-20 13:26:27 root: INFO: instruction count: 64: __C47etypeDecode.__m52INPLACE_DECODE_r_array_of_r_binary_TO_r_array_of_r_string 2021-08-20 13:26:27 root: INFO: instruction count: 33: __C47etypeDecode.__m53INPLACE_DECODE_o_binary_TO_o_string 2021-08-20 13:26:27 root: INFO: instruction count: 10: __C47etypeDecode.__m54INPLACE_DECODE_o_float64_TO_o_float64 2021-08-20 13:26:27 root: INFO: instruction count: 64: __C47etypeDecode.__m55INPLACE_DECODE_o_array_of_r_binary_TO_o_set_of_r_string 2021-08-20 13:26:27 root: INFO: instruction count: 477: __C47etypeDecode.__m56INPLACE_DECODE_r_struct_of_r_boolANDo_binaryANDo_array_of_o_int32ANDo_int32ANDo_array_of_o_int32ANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_binaryANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_int32ANDo_int32ANDo_int32END_TO_r_struct_of_r_boolANDo_stringANDo_array_of_o_int32ANDo_int32ANDo_array_of_o_int32ANDo_array_of_o_int32ANDo_array_of_o_stringANDo_array_of_o_stringANDo_stringANDo_array_of_o_int32ANDo_array_of_o_stringANDo_array_of_o_int32ANDo_array_of_o_stringANDo_array_of_o_stringANDo_array_of_o_stringANDo_int32ANDo_int32ANDo_int32END 2021-08-20 13:26:27 root: INFO: instruction count: 10: __C47etypeDecode.__m57INPLACE_DECODE_r_bool_TO_r_bool 2021-08-20 13:26:27 root: INFO: instruction count: 120: __C47etypeDecode.__m58INPLACE_DECODE_o_array_of_o_int32_TO_o_array_of_o_int32 2021-08-20 13:26:27 root: INFO: instruction count: 10: __C47etypeDecode.__m59INPLACE_DECODE_o_int32_TO_o_int32 2021-08-20 13:26:27 root: INFO: instruction count: 120: __C47etypeDecode.__m60INPLACE_DECODE_o_array_of_o_binary_TO_o_array_of_o_string 2021-08-20 13:26:27 root: INFO: instruction count: 123: __C47etypeDecode.__m56INPLACE_DECODE_r_struct_of_r_boolANDo_binaryANDo_array_of_o_int32ANDo_int32ANDo_array_of_o_int32ANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_binaryANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_int32ANDo_int32ANDo_int32END_TO_r_struct_of_r_boolANDo_stringANDo_array_of_o_int32ANDo_int32ANDo_array_of_o_int32ANDo_array_of_o_int32ANDo_array_of_o_stringANDo_array_of_o_stringANDo_stringANDo_array_of_o_int32ANDo_array_of_o_stringANDo_array_of_o_int32ANDo_array_of_o_stringANDo_array_of_o_stringANDo_array_of_o_stringANDo_int32ANDo_int32ANDo_int32ENDregion0_7 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C61__m56INPLACE_DECODE_r_struct_of_r_boolANDo_binaryANDo_array_of_o_int32ANDo_int32ANDo_array_of_o_int32ANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_binaryANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_int32ANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_array_of_o_binaryANDo_int32ANDo_int32ANDo_int32END_TO_r_struct_of_r_boolANDo_stringANDo_array_of_o_int32ANDo_int32ANDo_array_of_o_int32ANDo_array_of_o_int32ANDo_array_of_o_stringANDo_array_of_o_stringANDo_stringANDo_array_of_o_int32ANDo_array_of_o_stringANDo_array_of_o_int32ANDo_array_of_o_stringANDo_array_of_o_stringANDo_array_of_o_stringANDo_int32ANDo_int32ANDo_int32ENDSpills. 2021-08-20 13:26:27 root: INFO: decoder cache miss (1 hits, 5 misses, 0.167 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C67FSContainer. 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C67FSContainer. 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C68etypeDecode. 2021-08-20 13:26:27 root: INFO: instruction count: 5: __C68etypeDecode.apply 2021-08-20 13:26:27 root: INFO: instruction count: 23: __C68etypeDecode.__m69DECODE_r_struct_of_r_binaryEND_TO_o_struct_of_r_stringEND 2021-08-20 13:26:27 root: INFO: instruction count: 33: __C68etypeDecode.__m70INPLACE_DECODE_r_binary_TO_r_string 2021-08-20 13:26:27 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 2: (InsertFields (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32}})) 0) None) 2021-08-20 13:26:27 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 1: (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32}})) 0) 2021-08-20 13:26:27 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 1: (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32}})) 0) 2021-08-20 13:26:27 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 1: (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32}})) 0) 2021-08-20 13:26:27 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 1: (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32}})) 0) 2021-08-20 13:26:27 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 1: (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32}})) 0) 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C71FSContainer. 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C71FSContainer. 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C72Compiled. 2021-08-20 13:26:27 root: INFO: instruction count: 9: __C72Compiled.apply 2021-08-20 13:26:27 root: INFO: instruction count: 9: __C72Compiled.setPartitionIndex 2021-08-20 13:26:27 root: INFO: instruction count: 4: __C72Compiled.addPartitionRegion 2021-08-20 13:26:27 root: INFO: instruction count: 4: __C72Compiled.setPool 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C72Compiled.addFS 2021-08-20 13:26:27 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 13: (Coalesce (Let __iruid_84 (GetField __uid_3 (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},__uid_3:PCStruct{vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}}})) 1)) (InsertFields (SelectFields (locus alleles rsid qual filters info) (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},__uid_3:PCStruct{vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}}})) 1)) None (vep (GetField vep (Ref __iruid_84))) (vep_proc_id (GetField vep_proc_id (Ref __iruid_84))))) (Die Struct{locus:Locus(GRCh38),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{IMPRECISE:Boolean,SVTYPE:String,SVLEN:Array[Int32],END:Int32,CIPOS:Array[Int32],CIEND:Array[Int32],CIGAR:Array[String],MATEID:Array[String],EVENT:String,HOMLEN:Array[Int32],HOMSEQ:Array[String],SVINSLEN:Array[Int32],SVINSSEQ:Array[String],LEFT_SVINSSEQ:Array[String],RIGHT_SVINSSEQ:Array[String],BND_DEPTH:Int32,MATE_BND_DEPTH:Int32,JUNCTION_QUAL:Int32},vep:Array[String],vep_proc_id:Struct{part_idx:Int32,block_idx:Int32}} -1 (Str "Internal e..."))) 2021-08-20 13:26:27 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 13: (Coalesce (Let __iruid_91 (GetField __uid_3 (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},__uid_3:PCStruct{vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}}})) 1)) (InsertFields (SelectFields (locus alleles rsid qual filters info) (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},__uid_3:PCStruct{vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}}})) 1)) None (vep (GetField vep (Ref __iruid_91))) (vep_proc_id (GetField vep_proc_id (Ref __iruid_91))))) (Die Struct{locus:Locus(GRCh38),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{IMPRECISE:Boolean,SVTYPE:String,SVLEN:Array[Int32],END:Int32,CIPOS:Array[Int32],CIEND:Array[Int32],CIGAR:Array[String],MATEID:Array[String],EVENT:String,HOMLEN:Array[Int32],HOMSEQ:Array[String],SVINSLEN:Array[Int32],SVINSSEQ:Array[String],LEFT_SVINSSEQ:Array[String],RIGHT_SVINSSEQ:Array[String],BND_DEPTH:Int32,MATE_BND_DEPTH:Int32,JUNCTION_QUAL:Int32},vep:Array[String],vep_proc_id:Struct{part_idx:Int32,block_idx:Int32}} -1 (Str "Internal e..."))) 2021-08-20 13:26:27 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 13: (Coalesce (Let __iruid_91 (GetField __uid_3 (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},__uid_3:PCStruct{vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}}})) 1)) (InsertFields (SelectFields (locus alleles rsid qual filters info) (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},__uid_3:PCStruct{vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}}})) 1)) None (vep (GetField vep (Ref __iruid_91))) (vep_proc_id (GetField vep_proc_id (Ref __iruid_91))))) (Die Struct{locus:Locus(GRCh38),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{IMPRECISE:Boolean,SVTYPE:String,SVLEN:Array[Int32],END:Int32,CIPOS:Array[Int32],CIEND:Array[Int32],CIGAR:Array[String],MATEID:Array[String],EVENT:String,HOMLEN:Array[Int32],HOMSEQ:Array[String],SVINSLEN:Array[Int32],SVINSSEQ:Array[String],LEFT_SVINSSEQ:Array[String],RIGHT_SVINSSEQ:Array[String],BND_DEPTH:Int32,MATE_BND_DEPTH:Int32,JUNCTION_QUAL:Int32},vep:Array[String],vep_proc_id:Struct{part_idx:Int32,block_idx:Int32}} -1 (Str "Internal e..."))) 2021-08-20 13:26:27 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 13: (Coalesce (Let __iruid_94 (GetField __uid_3 (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},__uid_3:PCStruct{vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}}})) 1)) (InsertFields (SelectFields (locus alleles rsid qual filters info) (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},__uid_3:PCStruct{vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}}})) 1)) None (vep (GetField vep (Ref __iruid_94))) (vep_proc_id (GetField vep_proc_id (Ref __iruid_94))))) (Die Struct{locus:Locus(GRCh38),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{IMPRECISE:Boolean,SVTYPE:String,SVLEN:Array[Int32],END:Int32,CIPOS:Array[Int32],CIEND:Array[Int32],CIGAR:Array[String],MATEID:Array[String],EVENT:String,HOMLEN:Array[Int32],HOMSEQ:Array[String],SVINSLEN:Array[Int32],SVINSSEQ:Array[String],LEFT_SVINSSEQ:Array[String],RIGHT_SVINSSEQ:Array[String],BND_DEPTH:Int32,MATE_BND_DEPTH:Int32,JUNCTION_QUAL:Int32},vep:Array[String],vep_proc_id:Struct{part_idx:Int32,block_idx:Int32}} -1 (Str "Internal e..."))) 2021-08-20 13:26:27 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 13: (Coalesce (Let __iruid_94 (GetField __uid_3 (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},__uid_3:PCStruct{vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}}})) 1)) (InsertFields (SelectFields (locus alleles rsid qual filters info) (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},__uid_3:PCStruct{vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}}})) 1)) None (vep (GetField vep (Ref __iruid_94))) (vep_proc_id (GetField vep_proc_id (Ref __iruid_94))))) (Die Struct{locus:Locus(GRCh38),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{IMPRECISE:Boolean,SVTYPE:String,SVLEN:Array[Int32],END:Int32,CIPOS:Array[Int32],CIEND:Array[Int32],CIGAR:Array[String],MATEID:Array[String],EVENT:String,HOMLEN:Array[Int32],HOMSEQ:Array[String],SVINSLEN:Array[Int32],SVINSSEQ:Array[String],LEFT_SVINSSEQ:Array[String],RIGHT_SVINSSEQ:Array[String],BND_DEPTH:Int32,MATE_BND_DEPTH:Int32,JUNCTION_QUAL:Int32},vep:Array[String],vep_proc_id:Struct{part_idx:Int32,block_idx:Int32}} -1 (Str "Internal e..."))) 2021-08-20 13:26:27 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 13: (Coalesce (Let __iruid_97 (GetField __uid_3 (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},__uid_3:PCStruct{vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}}})) 1)) (InsertFields (SelectFields (locus alleles rsid qual filters info) (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},__uid_3:PCStruct{vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}}})) 1)) None (vep (GetField vep (Ref __iruid_97))) (vep_proc_id (GetField vep_proc_id (Ref __iruid_97))))) (Die Struct{locus:Locus(GRCh38),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{IMPRECISE:Boolean,SVTYPE:String,SVLEN:Array[Int32],END:Int32,CIPOS:Array[Int32],CIEND:Array[Int32],CIGAR:Array[String],MATEID:Array[String],EVENT:String,HOMLEN:Array[Int32],HOMSEQ:Array[String],SVINSLEN:Array[Int32],SVINSSEQ:Array[String],LEFT_SVINSSEQ:Array[String],RIGHT_SVINSSEQ:Array[String],BND_DEPTH:Int32,MATE_BND_DEPTH:Int32,JUNCTION_QUAL:Int32},vep:Array[String],vep_proc_id:Struct{part_idx:Int32,block_idx:Int32}} -1 (Str "Internal e..."))) 2021-08-20 13:26:27 root: INFO: encoder cache miss (0 hits, 3 misses, 0.000) 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C90FSContainer. 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C90FSContainer. 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C91etypeEncode. 2021-08-20 13:26:27 root: INFO: instruction count: 5: __C91etypeEncode.apply 2021-08-20 13:26:27 root: INFO: instruction count: 13: __C91etypeEncode.__m92ENCODE_o_tuple_of_r_stringEND_TO_r_struct_of_r_binaryEND 2021-08-20 13:26:27 root: INFO: instruction count: 18: __C91etypeEncode.__m93ENCODE_o_string_TO_r_binary 2021-08-20 13:26:27 MemoryStore: INFO: Block broadcast_2 stored as values in memory (estimated size 152.0 B, free 26.5 GiB) 2021-08-20 13:26:27 MemoryStore: INFO: Block broadcast_2_piece0 stored as bytes in memory (estimated size 115.0 B, free 26.5 GiB) 2021-08-20 13:26:27 BlockManagerInfo: INFO: Added broadcast_2_piece0 in memory on spark-master:32903 (size: 115.0 B, free: 26.5 GiB) 2021-08-20 13:26:27 SparkContext: INFO: Created broadcast 2 from broadcast at SparkBackend.scala:278 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C78FSContainer. 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C78FSContainer. 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C79Compiled. 2021-08-20 13:26:27 root: INFO: instruction count: 328: __C79Compiled.apply 2021-08-20 13:26:27 root: INFO: instruction count: 9: __C79Compiled.setPartitionIndex 2021-08-20 13:26:27 root: INFO: instruction count: 4: __C79Compiled.addPartitionRegion 2021-08-20 13:26:27 root: INFO: instruction count: 4: __C79Compiled.setPool 2021-08-20 13:26:27 root: INFO: instruction count: 3: __C79Compiled.addFS 2021-08-20 13:26:27 root: INFO: instruction count: 36: __C79Compiled.addLiterals 2021-08-20 13:26:27 root: INFO: instruction count: 23: __C79Compiled.__m88DECODE_r_struct_of_r_binaryEND_TO_o_tuple_of_r_stringEND 2021-08-20 13:26:27 root: INFO: instruction count: 33: __C79Compiled.__m89INPLACE_DECODE_r_binary_TO_r_string 2021-08-20 13:26:27 SparkContext: INFO: Starting job: runJob at ContextRDD.scala:362 2021-08-20 13:26:27 DAGScheduler: INFO: Got job 0 (runJob at ContextRDD.scala:362) with 1 output partitions 2021-08-20 13:26:27 DAGScheduler: INFO: Final stage: ResultStage 0 (runJob at ContextRDD.scala:362) 2021-08-20 13:26:27 DAGScheduler: INFO: Parents of final stage: List() 2021-08-20 13:26:27 DAGScheduler: INFO: Missing parents: List() 2021-08-20 13:26:27 DAGScheduler: INFO: Submitting ResultStage 0 (MapPartitionsRDD[25] at mapPartitions at ContextRDD.scala:250), which has no missing parents 2021-08-20 13:26:27 MemoryStore: INFO: Block broadcast_3 stored as values in memory (estimated size 209.0 KiB, free 26.5 GiB) 2021-08-20 13:26:27 MemoryStore: INFO: Block broadcast_3_piece0 stored as bytes in memory (estimated size 76.6 KiB, free 26.5 GiB) 2021-08-20 13:26:27 BlockManagerInfo: INFO: Added broadcast_3_piece0 in memory on spark-master:32903 (size: 76.6 KiB, free: 26.5 GiB) 2021-08-20 13:26:27 SparkContext: INFO: Created broadcast 3 from broadcast at DAGScheduler.scala:1383 2021-08-20 13:26:27 DAGScheduler: INFO: Submitting 1 missing tasks from ResultStage 0 (MapPartitionsRDD[25] at mapPartitions at ContextRDD.scala:250) (first 15 tasks are for partitions Vector(0)) 2021-08-20 13:26:27 TaskSchedulerImpl: INFO: Adding task set 0.0 with 1 tasks resource profile 0 2021-08-20 13:26:27 TaskSetManager: INFO: Starting task 0.0 in stage 0.0 (TID 0) (spark-master, executor driver, partition 0, PROCESS_LOCAL, 5409 bytes) taskResourceAssignments Map() 2021-08-20 13:26:27 Executor: INFO: Running task 0.0 in stage 0.0 (TID 0) 2021-08-20 13:26:28 root: INFO: RegionPool: initialized for thread 123: Executor task launch worker for task 0.0 in stage 0.0 (TID 0) 2021-08-20 13:26:28 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), regions.size = 3, 0 current java objects, thread 123: Executor task launch worker for task 0.0 in stage 0.0 (TID 0) 2021-08-20 13:26:28 root: INFO: RegionPool: REPORT_THRESHOLD: 512.0K allocated (512.0K blocks / 0 chunks), regions.size = 7, 0 current java objects, thread 123: Executor task launch worker for task 0.0 in stage 0.0 (TID 0) 2021-08-20 13:26:29 MemoryStore: INFO: Block rdd_11_0 stored as values in memory (estimated size 88.0 B, free 26.5 GiB) 2021-08-20 13:26:29 BlockManagerInfo: INFO: Added rdd_11_0 in memory on spark-master:32903 (size: 88.0 B, free: 26.5 GiB) 2021-08-20 13:26:29 root: INFO: TaskReport: stage=0, partition=0, attempt=0, peakBytes=720896, peakBytesReadable=704.00 KiB, chunks requested=0, cache hits=0 2021-08-20 13:26:29 root: INFO: RegionPool: FREE: 704.0K allocated (704.0K blocks / 0 chunks), regions.size = 11, 0 current java objects, thread 123: Executor task launch worker for task 0.0 in stage 0.0 (TID 0) 2021-08-20 13:26:29 Executor: INFO: 1 block locks were not released by task 0.0 in stage 0.0 (TID 0) [rdd_11_0] 2021-08-20 13:26:29 Executor: INFO: Finished task 0.0 in stage 0.0 (TID 0). 881 bytes result sent to driver 2021-08-20 13:26:29 TaskSetManager: INFO: Finished task 0.0 in stage 0.0 (TID 0) in 1470 ms on spark-master (executor driver) (1/1) 2021-08-20 13:26:29 TaskSchedulerImpl: INFO: Removed TaskSet 0.0, whose tasks have all completed, from pool 2021-08-20 13:26:29 DAGScheduler: INFO: ResultStage 0 (runJob at ContextRDD.scala:362) finished in 1.589 s 2021-08-20 13:26:29 DAGScheduler: INFO: Job 0 is finished. Cancelling potential speculative or zombie tasks for this job 2021-08-20 13:26:29 TaskSchedulerImpl: INFO: Killing all running tasks in stage 0: Stage finished 2021-08-20 13:26:29 DAGScheduler: INFO: Job 0 finished: runJob at ContextRDD.scala:362, took 1.664183 s 2021-08-20 13:26:29 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 93: (Coalesce (Let __iruid_85 (GetField vep_proc_id (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (Let __iruid_86 (GetField info (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (Let __iruid_87 (SelectFields (locus alleles rsid qual filters vep) (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (InsertFields (SelectFields () (Ref __iruid_87)) None (locus (ApplySpecial 1 showStr () String (GetField locus (Ref __iruid_87)))) (alleles (ApplySpecial 2 showStr () String (GetField alleles (Ref __iruid_87)))) (rsid (ApplySpecial 3 showStr () String (GetField rsid (Ref __iruid_87)))) (qual (ApplySpecial 4 showStr () String (GetField qual (Ref __iruid_87)))) (filters (ApplySpecial 5 showStr () String (GetField filters (Ref __iruid_87)))) (`info.IMPRECISE` (ApplySpecial 6 showStr () String (GetField IMPRECISE (Ref __iruid_86)))) (`info.SVTYPE` (ApplySpecial 7 showStr () String (GetField SVTYPE (Ref __iruid_86)))) (`info.SVLEN` (ApplySpecial 8 showStr () String (GetField SVLEN (Ref __iruid_86)))) (`info.END` (ApplySpecial 9 showStr () String (GetField END (Ref __iruid_86)))) (`info.CIPOS` (ApplySpecial 10 showStr () String (GetField CIPOS (Ref __iruid_86)))) (`info.CIEND` (ApplySpecial 11 showStr () String (GetField CIEND (Ref __iruid_86)))) (`info.CIGAR` (ApplySpecial 12 showStr () String (GetField CIGAR (Ref __iruid_86)))) (`info.MATEID` (ApplySpecial 13 showStr () String (GetField MATEID (Ref __iruid_86)))) (`info.EVENT` (ApplySpecial 14 showStr () String (GetField EVENT (Ref __iruid_86)))) (`info.HOMLEN` (ApplySpecial 15 showStr () String (GetField HOMLEN (Ref __iruid_86)))) (`info.HOMSEQ` (ApplySpecial 16 showStr () String (GetField HOMSEQ (Ref __iruid_86)))) (`info.SVINSLEN` (ApplySpecial 17 showStr () String (GetField SVINSLEN (Ref __iruid_86)))) (`info.SVINSSEQ` (ApplySpecial 18 showStr () String (GetField SVINSSEQ (Ref __iruid_86)))) (`info.LEFT_SVINSSEQ` (ApplySpecial 19 showStr () String (GetField LEFT_SVINSSEQ (Ref __iruid_86)))) (`info.RIGHT_SVINSSEQ` (ApplySpecial 20 showStr () String (GetField RIGHT_SVINSSEQ (Ref __iruid_86)))) (`info.BND_DEPTH` (ApplySpecial 21 showStr () String (GetField BND_DEPTH (Ref __iruid_86)))) (`info.MATE_BND_DEPTH` (ApplySpecial 22 showStr () String (GetField MATE_BND_DEPTH (Ref __iruid_86)))) (`info.JUNCTION_QUAL` (ApplySpecial 23 showStr () String (GetField JUNCTION_QUAL (Ref __iruid_86)))) (vep (ApplySpecial 24 showStr () String (GetField vep (Ref __iruid_87)))) (`vep_proc_id.part_idx` (ApplySpecial 25 showStr () String (GetField part_idx (Ref __iruid_85)))) (`vep_proc_id.block_idx` (ApplySpecial 26 showStr () String (GetField block_idx (Ref __iruid_85)))))))) (Die Struct{locus:String,alleles:String,rsid:String,qual:String,filters:String,`info.IMPRECISE`:String,`info.SVTYPE`:String,`info.SVLEN`:String,`info.END`:String,`info.CIPOS`:String,`info.CIEND`:String,`info.CIGAR`:String,`info.MATEID`:String,`info.EVENT`:String,`info.HOMLEN`:String,`info.HOMSEQ`:String,`info.SVINSLEN`:String,`info.SVINSSEQ`:String,`info.LEFT_SVINSSEQ`:String,`info.RIGHT_SVINSSEQ`:String,`info.BND_DEPTH`:String,`info.MATE_BND_DEPTH`:String,`info.JUNCTION_QUAL`:String,vep:String,`vep_proc_id.part_idx`:String,`vep_proc_id.block_idx`:String} -1 (Str "Internal e..."))) 2021-08-20 13:26:29 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 93: (Coalesce (Let __iruid_105 (GetField vep_proc_id (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (Let __iruid_106 (GetField info (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (Let __iruid_107 (SelectFields (locus alleles rsid qual filters vep) (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (InsertFields (SelectFields () (Ref __iruid_107)) None (locus (ApplySpecial 1 showStr () String (GetField locus (Ref __iruid_107)))) (alleles (ApplySpecial 2 showStr () String (GetField alleles (Ref __iruid_107)))) (rsid (ApplySpecial 3 showStr () String (GetField rsid (Ref __iruid_107)))) (qual (ApplySpecial 4 showStr () String (GetField qual (Ref __iruid_107)))) (filters (ApplySpecial 5 showStr () String (GetField filters (Ref __iruid_107)))) (`info.IMPRECISE` (ApplySpecial 6 showStr () String (GetField IMPRECISE (Ref __iruid_106)))) (`info.SVTYPE` (ApplySpecial 7 showStr () String (GetField SVTYPE (Ref __iruid_106)))) (`info.SVLEN` (ApplySpecial 8 showStr () String (GetField SVLEN (Ref __iruid_106)))) (`info.END` (ApplySpecial 9 showStr () String (GetField END (Ref __iruid_106)))) (`info.CIPOS` (ApplySpecial 10 showStr () String (GetField CIPOS (Ref __iruid_106)))) (`info.CIEND` (ApplySpecial 11 showStr () String (GetField CIEND (Ref __iruid_106)))) (`info.CIGAR` (ApplySpecial 12 showStr () String (GetField CIGAR (Ref __iruid_106)))) (`info.MATEID` (ApplySpecial 13 showStr () String (GetField MATEID (Ref __iruid_106)))) (`info.EVENT` (ApplySpecial 14 showStr () String (GetField EVENT (Ref __iruid_106)))) (`info.HOMLEN` (ApplySpecial 15 showStr () String (GetField HOMLEN (Ref __iruid_106)))) (`info.HOMSEQ` (ApplySpecial 16 showStr () String (GetField HOMSEQ (Ref __iruid_106)))) (`info.SVINSLEN` (ApplySpecial 17 showStr () String (GetField SVINSLEN (Ref __iruid_106)))) (`info.SVINSSEQ` (ApplySpecial 18 showStr () String (GetField SVINSSEQ (Ref __iruid_106)))) (`info.LEFT_SVINSSEQ` (ApplySpecial 19 showStr () String (GetField LEFT_SVINSSEQ (Ref __iruid_106)))) (`info.RIGHT_SVINSSEQ` (ApplySpecial 20 showStr () String (GetField RIGHT_SVINSSEQ (Ref __iruid_106)))) (`info.BND_DEPTH` (ApplySpecial 21 showStr () String (GetField BND_DEPTH (Ref __iruid_106)))) (`info.MATE_BND_DEPTH` (ApplySpecial 22 showStr () String (GetField MATE_BND_DEPTH (Ref __iruid_106)))) (`info.JUNCTION_QUAL` (ApplySpecial 23 showStr () String (GetField JUNCTION_QUAL (Ref __iruid_106)))) (vep (ApplySpecial 24 showStr () String (GetField vep (Ref __iruid_107)))) (`vep_proc_id.part_idx` (ApplySpecial 25 showStr () String (GetField part_idx (Ref __iruid_105)))) (`vep_proc_id.block_idx` (ApplySpecial 26 showStr () String (GetField block_idx (Ref __iruid_105)))))))) (Die Struct{locus:String,alleles:String,rsid:String,qual:String,filters:String,`info.IMPRECISE`:String,`info.SVTYPE`:String,`info.SVLEN`:String,`info.END`:String,`info.CIPOS`:String,`info.CIEND`:String,`info.CIGAR`:String,`info.MATEID`:String,`info.EVENT`:String,`info.HOMLEN`:String,`info.HOMSEQ`:String,`info.SVINSLEN`:String,`info.SVINSSEQ`:String,`info.LEFT_SVINSSEQ`:String,`info.RIGHT_SVINSSEQ`:String,`info.BND_DEPTH`:String,`info.MATE_BND_DEPTH`:String,`info.JUNCTION_QUAL`:String,vep:String,`vep_proc_id.part_idx`:String,`vep_proc_id.block_idx`:String} -1 (Str "Internal e..."))) 2021-08-20 13:26:29 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 93: (Coalesce (Let __iruid_105 (GetField vep_proc_id (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (Let __iruid_106 (GetField info (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (Let __iruid_107 (SelectFields (locus alleles rsid qual filters vep) (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (InsertFields (SelectFields () (Ref __iruid_107)) None (locus (ApplySpecial 1 showStr () String (GetField locus (Ref __iruid_107)))) (alleles (ApplySpecial 2 showStr () String (GetField alleles (Ref __iruid_107)))) (rsid (ApplySpecial 3 showStr () String (GetField rsid (Ref __iruid_107)))) (qual (ApplySpecial 4 showStr () String (GetField qual (Ref __iruid_107)))) (filters (ApplySpecial 5 showStr () String (GetField filters (Ref __iruid_107)))) (`info.IMPRECISE` (ApplySpecial 6 showStr () String (GetField IMPRECISE (Ref __iruid_106)))) (`info.SVTYPE` (ApplySpecial 7 showStr () String (GetField SVTYPE (Ref __iruid_106)))) (`info.SVLEN` (ApplySpecial 8 showStr () String (GetField SVLEN (Ref __iruid_106)))) (`info.END` (ApplySpecial 9 showStr () String (GetField END (Ref __iruid_106)))) (`info.CIPOS` (ApplySpecial 10 showStr () String (GetField CIPOS (Ref __iruid_106)))) (`info.CIEND` (ApplySpecial 11 showStr () String (GetField CIEND (Ref __iruid_106)))) (`info.CIGAR` (ApplySpecial 12 showStr () String (GetField CIGAR (Ref __iruid_106)))) (`info.MATEID` (ApplySpecial 13 showStr () String (GetField MATEID (Ref __iruid_106)))) (`info.EVENT` (ApplySpecial 14 showStr () String (GetField EVENT (Ref __iruid_106)))) (`info.HOMLEN` (ApplySpecial 15 showStr () String (GetField HOMLEN (Ref __iruid_106)))) (`info.HOMSEQ` (ApplySpecial 16 showStr () String (GetField HOMSEQ (Ref __iruid_106)))) (`info.SVINSLEN` (ApplySpecial 17 showStr () String (GetField SVINSLEN (Ref __iruid_106)))) (`info.SVINSSEQ` (ApplySpecial 18 showStr () String (GetField SVINSSEQ (Ref __iruid_106)))) (`info.LEFT_SVINSSEQ` (ApplySpecial 19 showStr () String (GetField LEFT_SVINSSEQ (Ref __iruid_106)))) (`info.RIGHT_SVINSSEQ` (ApplySpecial 20 showStr () String (GetField RIGHT_SVINSSEQ (Ref __iruid_106)))) (`info.BND_DEPTH` (ApplySpecial 21 showStr () String (GetField BND_DEPTH (Ref __iruid_106)))) (`info.MATE_BND_DEPTH` (ApplySpecial 22 showStr () String (GetField MATE_BND_DEPTH (Ref __iruid_106)))) (`info.JUNCTION_QUAL` (ApplySpecial 23 showStr () String (GetField JUNCTION_QUAL (Ref __iruid_106)))) (vep (ApplySpecial 24 showStr () String (GetField vep (Ref __iruid_107)))) (`vep_proc_id.part_idx` (ApplySpecial 25 showStr () String (GetField part_idx (Ref __iruid_105)))) (`vep_proc_id.block_idx` (ApplySpecial 26 showStr () String (GetField block_idx (Ref __iruid_105)))))))) (Die Struct{locus:String,alleles:String,rsid:String,qual:String,filters:String,`info.IMPRECISE`:String,`info.SVTYPE`:String,`info.SVLEN`:String,`info.END`:String,`info.CIPOS`:String,`info.CIEND`:String,`info.CIGAR`:String,`info.MATEID`:String,`info.EVENT`:String,`info.HOMLEN`:String,`info.HOMSEQ`:String,`info.SVINSLEN`:String,`info.SVINSSEQ`:String,`info.LEFT_SVINSSEQ`:String,`info.RIGHT_SVINSSEQ`:String,`info.BND_DEPTH`:String,`info.MATE_BND_DEPTH`:String,`info.JUNCTION_QUAL`:String,vep:String,`vep_proc_id.part_idx`:String,`vep_proc_id.block_idx`:String} -1 (Str "Internal e..."))) 2021-08-20 13:26:29 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 93: (Coalesce (Let __iruid_114 (GetField vep_proc_id (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (Let __iruid_115 (GetField info (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (Let __iruid_116 (SelectFields (locus alleles rsid qual filters vep) (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (InsertFields (SelectFields () (Ref __iruid_116)) None (locus (ApplySpecial 1 showStr () String (GetField locus (Ref __iruid_116)))) (alleles (ApplySpecial 2 showStr () String (GetField alleles (Ref __iruid_116)))) (rsid (ApplySpecial 3 showStr () String (GetField rsid (Ref __iruid_116)))) (qual (ApplySpecial 4 showStr () String (GetField qual (Ref __iruid_116)))) (filters (ApplySpecial 5 showStr () String (GetField filters (Ref __iruid_116)))) (`info.IMPRECISE` (ApplySpecial 6 showStr () String (GetField IMPRECISE (Ref __iruid_115)))) (`info.SVTYPE` (ApplySpecial 7 showStr () String (GetField SVTYPE (Ref __iruid_115)))) (`info.SVLEN` (ApplySpecial 8 showStr () String (GetField SVLEN (Ref __iruid_115)))) (`info.END` (ApplySpecial 9 showStr () String (GetField END (Ref __iruid_115)))) (`info.CIPOS` (ApplySpecial 10 showStr () String (GetField CIPOS (Ref __iruid_115)))) (`info.CIEND` (ApplySpecial 11 showStr () String (GetField CIEND (Ref __iruid_115)))) (`info.CIGAR` (ApplySpecial 12 showStr () String (GetField CIGAR (Ref __iruid_115)))) (`info.MATEID` (ApplySpecial 13 showStr () String (GetField MATEID (Ref __iruid_115)))) (`info.EVENT` (ApplySpecial 14 showStr () String (GetField EVENT (Ref __iruid_115)))) (`info.HOMLEN` (ApplySpecial 15 showStr () String (GetField HOMLEN (Ref __iruid_115)))) (`info.HOMSEQ` (ApplySpecial 16 showStr () String (GetField HOMSEQ (Ref __iruid_115)))) (`info.SVINSLEN` (ApplySpecial 17 showStr () String (GetField SVINSLEN (Ref __iruid_115)))) (`info.SVINSSEQ` (ApplySpecial 18 showStr () String (GetField SVINSSEQ (Ref __iruid_115)))) (`info.LEFT_SVINSSEQ` (ApplySpecial 19 showStr () String (GetField LEFT_SVINSSEQ (Ref __iruid_115)))) (`info.RIGHT_SVINSSEQ` (ApplySpecial 20 showStr () String (GetField RIGHT_SVINSSEQ (Ref __iruid_115)))) (`info.BND_DEPTH` (ApplySpecial 21 showStr () String (GetField BND_DEPTH (Ref __iruid_115)))) (`info.MATE_BND_DEPTH` (ApplySpecial 22 showStr () String (GetField MATE_BND_DEPTH (Ref __iruid_115)))) (`info.JUNCTION_QUAL` (ApplySpecial 23 showStr () String (GetField JUNCTION_QUAL (Ref __iruid_115)))) (vep (ApplySpecial 24 showStr () String (GetField vep (Ref __iruid_116)))) (`vep_proc_id.part_idx` (ApplySpecial 25 showStr () String (GetField part_idx (Ref __iruid_114)))) (`vep_proc_id.block_idx` (ApplySpecial 26 showStr () String (GetField block_idx (Ref __iruid_114)))))))) (Die Struct{locus:String,alleles:String,rsid:String,qual:String,filters:String,`info.IMPRECISE`:String,`info.SVTYPE`:String,`info.SVLEN`:String,`info.END`:String,`info.CIPOS`:String,`info.CIEND`:String,`info.CIGAR`:String,`info.MATEID`:String,`info.EVENT`:String,`info.HOMLEN`:String,`info.HOMSEQ`:String,`info.SVINSLEN`:String,`info.SVINSSEQ`:String,`info.LEFT_SVINSSEQ`:String,`info.RIGHT_SVINSSEQ`:String,`info.BND_DEPTH`:String,`info.MATE_BND_DEPTH`:String,`info.JUNCTION_QUAL`:String,vep:String,`vep_proc_id.part_idx`:String,`vep_proc_id.block_idx`:String} -1 (Str "Internal e..."))) 2021-08-20 13:26:29 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 93: (Coalesce (Let __iruid_114 (GetField vep_proc_id (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (Let __iruid_115 (GetField info (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (Let __iruid_116 (SelectFields (locus alleles rsid qual filters vep) (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (InsertFields (SelectFields () (Ref __iruid_116)) None (locus (ApplySpecial 1 showStr () String (GetField locus (Ref __iruid_116)))) (alleles (ApplySpecial 2 showStr () String (GetField alleles (Ref __iruid_116)))) (rsid (ApplySpecial 3 showStr () String (GetField rsid (Ref __iruid_116)))) (qual (ApplySpecial 4 showStr () String (GetField qual (Ref __iruid_116)))) (filters (ApplySpecial 5 showStr () String (GetField filters (Ref __iruid_116)))) (`info.IMPRECISE` (ApplySpecial 6 showStr () String (GetField IMPRECISE (Ref __iruid_115)))) (`info.SVTYPE` (ApplySpecial 7 showStr () String (GetField SVTYPE (Ref __iruid_115)))) (`info.SVLEN` (ApplySpecial 8 showStr () String (GetField SVLEN (Ref __iruid_115)))) (`info.END` (ApplySpecial 9 showStr () String (GetField END (Ref __iruid_115)))) (`info.CIPOS` (ApplySpecial 10 showStr () String (GetField CIPOS (Ref __iruid_115)))) (`info.CIEND` (ApplySpecial 11 showStr () String (GetField CIEND (Ref __iruid_115)))) (`info.CIGAR` (ApplySpecial 12 showStr () String (GetField CIGAR (Ref __iruid_115)))) (`info.MATEID` (ApplySpecial 13 showStr () String (GetField MATEID (Ref __iruid_115)))) (`info.EVENT` (ApplySpecial 14 showStr () String (GetField EVENT (Ref __iruid_115)))) (`info.HOMLEN` (ApplySpecial 15 showStr () String (GetField HOMLEN (Ref __iruid_115)))) (`info.HOMSEQ` (ApplySpecial 16 showStr () String (GetField HOMSEQ (Ref __iruid_115)))) (`info.SVINSLEN` (ApplySpecial 17 showStr () String (GetField SVINSLEN (Ref __iruid_115)))) (`info.SVINSSEQ` (ApplySpecial 18 showStr () String (GetField SVINSSEQ (Ref __iruid_115)))) (`info.LEFT_SVINSSEQ` (ApplySpecial 19 showStr () String (GetField LEFT_SVINSSEQ (Ref __iruid_115)))) (`info.RIGHT_SVINSSEQ` (ApplySpecial 20 showStr () String (GetField RIGHT_SVINSSEQ (Ref __iruid_115)))) (`info.BND_DEPTH` (ApplySpecial 21 showStr () String (GetField BND_DEPTH (Ref __iruid_115)))) (`info.MATE_BND_DEPTH` (ApplySpecial 22 showStr () String (GetField MATE_BND_DEPTH (Ref __iruid_115)))) (`info.JUNCTION_QUAL` (ApplySpecial 23 showStr () String (GetField JUNCTION_QUAL (Ref __iruid_115)))) (vep (ApplySpecial 24 showStr () String (GetField vep (Ref __iruid_116)))) (`vep_proc_id.part_idx` (ApplySpecial 25 showStr () String (GetField part_idx (Ref __iruid_114)))) (`vep_proc_id.block_idx` (ApplySpecial 26 showStr () String (GetField block_idx (Ref __iruid_114)))))))) (Die Struct{locus:String,alleles:String,rsid:String,qual:String,filters:String,`info.IMPRECISE`:String,`info.SVTYPE`:String,`info.SVLEN`:String,`info.END`:String,`info.CIPOS`:String,`info.CIEND`:String,`info.CIGAR`:String,`info.MATEID`:String,`info.EVENT`:String,`info.HOMLEN`:String,`info.HOMSEQ`:String,`info.SVINSLEN`:String,`info.SVINSSEQ`:String,`info.LEFT_SVINSSEQ`:String,`info.RIGHT_SVINSSEQ`:String,`info.BND_DEPTH`:String,`info.MATE_BND_DEPTH`:String,`info.JUNCTION_QUAL`:String,vep:String,`vep_proc_id.part_idx`:String,`vep_proc_id.block_idx`:String} -1 (Str "Internal e..."))) 2021-08-20 13:26:29 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 93: (Coalesce (Let __iruid_123 (GetField vep_proc_id (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (Let __iruid_124 (GetField info (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (Let __iruid_125 (SelectFields (locus alleles rsid qual filters vep) (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:+PCLocus(GRCh38),alleles:+PCArray[+PCString],rsid:PCString,qual:PFloat64,filters:PCSet[+PCString],info:+PCStruct{IMPRECISE:+PBoolean,SVTYPE:PCString,SVLEN:PCArray[PInt32],END:PInt32,CIPOS:PCArray[PInt32],CIEND:PCArray[PInt32],CIGAR:PCArray[PCString],MATEID:PCArray[PCString],EVENT:PCString,HOMLEN:PCArray[PInt32],HOMSEQ:PCArray[PCString],SVINSLEN:PCArray[PInt32],SVINSSEQ:PCArray[PCString],LEFT_SVINSSEQ:PCArray[PCString],RIGHT_SVINSSEQ:PCArray[PCString],BND_DEPTH:PInt32,MATE_BND_DEPTH:PInt32,JUNCTION_QUAL:PInt32},vep:PCArray[PCString],vep_proc_id:PCStruct{part_idx:PInt32,block_idx:PInt32}})) 1)) (InsertFields (SelectFields () (Ref __iruid_125)) None (locus (ApplySpecial 1 showStr () String (GetField locus (Ref __iruid_125)))) (alleles (ApplySpecial 2 showStr () String (GetField alleles (Ref __iruid_125)))) (rsid (ApplySpecial 3 showStr () String (GetField rsid (Ref __iruid_125)))) (qual (ApplySpecial 4 showStr () String (GetField qual (Ref __iruid_125)))) (filters (ApplySpecial 5 showStr () String (GetField filters (Ref __iruid_125)))) (`info.IMPRECISE` (ApplySpecial 6 showStr () String (GetField IMPRECISE (Ref __iruid_124)))) (`info.SVTYPE` (ApplySpecial 7 showStr () String (GetField SVTYPE (Ref __iruid_124)))) (`info.SVLEN` (ApplySpecial 8 showStr () String (GetField SVLEN (Ref __iruid_124)))) (`info.END` (ApplySpecial 9 showStr () String (GetField END (Ref __iruid_124)))) (`info.CIPOS` (ApplySpecial 10 showStr () String (GetField CIPOS (Ref __iruid_124)))) (`info.CIEND` (ApplySpecial 11 showStr () String (GetField CIEND (Ref __iruid_124)))) (`info.CIGAR` (ApplySpecial 12 showStr () String (GetField CIGAR (Ref __iruid_124)))) (`info.MATEID` (ApplySpecial 13 showStr () String (GetField MATEID (Ref __iruid_124)))) (`info.EVENT` (ApplySpecial 14 showStr () String (GetField EVENT (Ref __iruid_124)))) (`info.HOMLEN` (ApplySpecial 15 showStr () String (GetField HOMLEN (Ref __iruid_124)))) (`info.HOMSEQ` (ApplySpecial 16 showStr () String (GetField HOMSEQ (Ref __iruid_124)))) (`info.SVINSLEN` (ApplySpecial 17 showStr () String (GetField SVINSLEN (Ref __iruid_124)))) (`info.SVINSSEQ` (ApplySpecial 18 showStr () String (GetField SVINSSEQ (Ref __iruid_124)))) (`info.LEFT_SVINSSEQ` (ApplySpecial 19 showStr () String (GetField LEFT_SVINSSEQ (Ref __iruid_124)))) (`info.RIGHT_SVINSSEQ` (ApplySpecial 20 showStr () String (GetField RIGHT_SVINSSEQ (Ref __iruid_124)))) (`info.BND_DEPTH` (ApplySpecial 21 showStr () String (GetField BND_DEPTH (Ref __iruid_124)))) (`info.MATE_BND_DEPTH` (ApplySpecial 22 showStr () String (GetField MATE_BND_DEPTH (Ref __iruid_124)))) (`info.JUNCTION_QUAL` (ApplySpecial 23 showStr () String (GetField JUNCTION_QUAL (Ref __iruid_124)))) (vep (ApplySpecial 24 showStr () String (GetField vep (Ref __iruid_125)))) (`vep_proc_id.part_idx` (ApplySpecial 25 showStr () String (GetField part_idx (Ref __iruid_123)))) (`vep_proc_id.block_idx` (ApplySpecial 26 showStr () String (GetField block_idx (Ref __iruid_123)))))))) (Die Struct{locus:String,alleles:String,rsid:String,qual:String,filters:String,`info.IMPRECISE`:String,`info.SVTYPE`:String,`info.SVLEN`:String,`info.END`:String,`info.CIPOS`:String,`info.CIEND`:String,`info.CIGAR`:String,`info.MATEID`:String,`info.EVENT`:String,`info.HOMLEN`:String,`info.HOMSEQ`:String,`info.SVINSLEN`:String,`info.SVINSSEQ`:String,`info.LEFT_SVINSSEQ`:String,`info.RIGHT_SVINSSEQ`:String,`info.BND_DEPTH`:String,`info.MATE_BND_DEPTH`:String,`info.JUNCTION_QUAL`:String,vep:String,`vep_proc_id.part_idx`:String,`vep_proc_id.block_idx`:String} -1 (Str "Internal e..."))) 2021-08-20 13:26:29 root: INFO: encoder cache hit 2021-08-20 13:26:29 MemoryStore: INFO: Block broadcast_4 stored as values in memory (estimated size 152.0 B, free 26.5 GiB) 2021-08-20 13:26:29 MemoryStore: INFO: Block broadcast_4_piece0 stored as bytes in memory (estimated size 115.0 B, free 26.5 GiB) 2021-08-20 13:26:29 BlockManagerInfo: INFO: Added broadcast_4_piece0 in memory on spark-master:32903 (size: 115.0 B, free: 26.5 GiB) 2021-08-20 13:26:29 SparkContext: INFO: Created broadcast 4 from broadcast at SparkBackend.scala:278 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C94FSContainer. 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C94FSContainer. 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C95Compiled. 2021-08-20 13:26:29 root: INFO: instruction count: 349: __C95Compiled.apply 2021-08-20 13:26:29 root: INFO: instruction count: 405: __C95Compiled.__m102split_InsertFields 2021-08-20 13:26:29 root: INFO: instruction count: 23: __C95Compiled.__m107setup_null 2021-08-20 13:26:29 root: INFO: instruction count: 12: __C95Compiled.__m112setup_null 2021-08-20 13:26:29 root: INFO: instruction count: 12: __C95Compiled.__m115setup_null 2021-08-20 13:26:29 root: INFO: instruction count: 12: __C95Compiled.__m120setup_null 2021-08-20 13:26:29 root: INFO: instruction count: 12: __C95Compiled.__m125setup_null 2021-08-20 13:26:29 root: INFO: instruction count: 12: __C95Compiled.__m130setup_null 2021-08-20 13:26:29 root: INFO: instruction count: 12: __C95Compiled.__m133setup_null 2021-08-20 13:26:29 root: INFO: instruction count: 12: __C95Compiled.__m138setup_null 2021-08-20 13:26:29 root: INFO: instruction count: 12: __C95Compiled.__m145setup_null 2021-08-20 13:26:29 root: INFO: instruction count: 12: __C95Compiled.__m148setup_null 2021-08-20 13:26:29 root: INFO: instruction count: 12: __C95Compiled.__m153setup_null 2021-08-20 13:26:29 root: INFO: instruction count: 12: __C95Compiled.__m162setup_null 2021-08-20 13:26:29 root: INFO: instruction count: 9: __C95Compiled.setPartitionIndex 2021-08-20 13:26:29 root: INFO: instruction count: 4: __C95Compiled.addPartitionRegion 2021-08-20 13:26:29 root: INFO: instruction count: 4: __C95Compiled.setPool 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C95Compiled.addFS 2021-08-20 13:26:29 root: INFO: instruction count: 41: __C95Compiled.addReferenceGenomes 2021-08-20 13:26:29 root: INFO: instruction count: 36: __C95Compiled.addLiterals 2021-08-20 13:26:29 root: INFO: instruction count: 23: __C95Compiled.__m224DECODE_r_struct_of_r_binaryEND_TO_o_tuple_of_r_stringEND 2021-08-20 13:26:29 root: INFO: instruction count: 33: __C95Compiled.__m225INPLACE_DECODE_r_binary_TO_r_string 2021-08-20 13:26:29 root: INFO: instruction count: 136: __C95Compiled.__m102split_InsertFieldsregion29_34 2021-08-20 13:26:29 root: INFO: instruction count: 135: __C95Compiled.__m102split_InsertFieldsregion0_5 2021-08-20 13:26:29 root: INFO: instruction count: 101: __C95Compiled.__m102split_InsertFieldsregion94_100 2021-08-20 13:26:29 root: INFO: instruction count: 101: __C95Compiled.__m102split_InsertFieldsregion35_41 2021-08-20 13:26:29 root: INFO: instruction count: 99: __C95Compiled.__m102split_InsertFieldsregion6_12 2021-08-20 13:26:29 root: INFO: instruction count: 98: __C95Compiled.__m102split_InsertFieldsregion146_154 2021-08-20 13:26:29 root: INFO: instruction count: 98: __C95Compiled.__m102split_InsertFieldsregion137_145 2021-08-20 13:26:29 root: INFO: instruction count: 98: __C95Compiled.__m102split_InsertFieldsregion128_136 2021-08-20 13:26:29 root: INFO: instruction count: 98: __C95Compiled.__m102split_InsertFieldsregion119_127 2021-08-20 13:26:29 root: INFO: instruction count: 98: __C95Compiled.__m102split_InsertFieldsregion110_118 2021-08-20 13:26:29 root: INFO: instruction count: 98: __C95Compiled.__m102split_InsertFieldsregion101_109 2021-08-20 13:26:29 root: INFO: instruction count: 98: __C95Compiled.__m102split_InsertFieldsregion76_84 2021-08-20 13:26:29 root: INFO: instruction count: 98: __C95Compiled.__m102split_InsertFieldsregion67_75 2021-08-20 13:26:29 root: INFO: instruction count: 98: __C95Compiled.__m102split_InsertFieldsregion85_93 2021-08-20 13:26:29 root: INFO: instruction count: 98: __C95Compiled.__m102split_InsertFieldsregion58_66 2021-08-20 13:26:29 root: INFO: instruction count: 98: __C95Compiled.__m102split_InsertFieldsregion42_50 2021-08-20 13:26:29 root: INFO: instruction count: 96: __C95Compiled.__m102split_InsertFieldsregion20_28 2021-08-20 13:26:29 root: INFO: instruction count: 87: __C95Compiled.__m102split_InsertFieldsregion193_200 2021-08-20 13:26:29 root: INFO: instruction count: 87: __C95Compiled.__m102split_InsertFieldsregion185_192 2021-08-20 13:26:29 root: INFO: instruction count: 87: __C95Compiled.__m102split_InsertFieldsregion176_184 2021-08-20 13:26:29 root: INFO: instruction count: 88: __C95Compiled.__m102split_InsertFieldsregion169_175 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C226__m102split_InsertFieldsSpills. 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C104RGContainer_GRCh38. 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C104RGContainer_GRCh38. 2021-08-20 13:26:29 root: INFO: encoder cache miss (1 hits, 4 misses, 0.200) 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C265FSContainer. 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C265FSContainer. 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C266etypeEncode. 2021-08-20 13:26:29 root: INFO: instruction count: 5: __C266etypeEncode.apply 2021-08-20 13:26:29 root: INFO: instruction count: 263: __C266etypeEncode.__m267ENCODE_o_struct_of_r_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringEND_TO_r_struct_of_r_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryEND 2021-08-20 13:26:29 root: INFO: instruction count: 18: __C266etypeEncode.__m268ENCODE_o_string_TO_r_binary 2021-08-20 13:26:29 SparkContext: INFO: Starting job: collect at RVD.scala:763 2021-08-20 13:26:29 DAGScheduler: INFO: Got job 1 (collect at RVD.scala:763) with 1 output partitions 2021-08-20 13:26:29 DAGScheduler: INFO: Final stage: ResultStage 1 (collect at RVD.scala:763) 2021-08-20 13:26:29 DAGScheduler: INFO: Parents of final stage: List() 2021-08-20 13:26:29 DAGScheduler: INFO: Missing parents: List() 2021-08-20 13:26:29 DAGScheduler: INFO: Submitting ResultStage 1 (MapPartitionsRDD[31] at mapPartitions at ContextRDD.scala:168), which has no missing parents 2021-08-20 13:26:29 MemoryStore: INFO: Block broadcast_5 stored as values in memory (estimated size 454.9 KiB, free 26.5 GiB) 2021-08-20 13:26:29 MemoryStore: INFO: Block broadcast_5_piece0 stored as bytes in memory (estimated size 214.6 KiB, free 26.5 GiB) 2021-08-20 13:26:29 BlockManagerInfo: INFO: Added broadcast_5_piece0 in memory on spark-master:32903 (size: 214.6 KiB, free: 26.5 GiB) 2021-08-20 13:26:29 SparkContext: INFO: Created broadcast 5 from broadcast at DAGScheduler.scala:1383 2021-08-20 13:26:29 DAGScheduler: INFO: Submitting 1 missing tasks from ResultStage 1 (MapPartitionsRDD[31] at mapPartitions at ContextRDD.scala:168) (first 15 tasks are for partitions Vector(0)) 2021-08-20 13:26:29 TaskSchedulerImpl: INFO: Adding task set 1.0 with 1 tasks resource profile 0 2021-08-20 13:26:29 TaskSetManager: INFO: Starting task 0.0 in stage 1.0 (TID 1) (spark-master, executor driver, partition 0, PROCESS_LOCAL, 5503 bytes) taskResourceAssignments Map() 2021-08-20 13:26:29 Executor: INFO: Running task 0.0 in stage 1.0 (TID 1) 2021-08-20 13:26:29 root: INFO: RegionPool: initialized for thread 123: Executor task launch worker for task 0.0 in stage 1.0 (TID 1) 2021-08-20 13:26:29 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), regions.size = 3, 0 current java objects, thread 123: Executor task launch worker for task 0.0 in stage 1.0 (TID 1) 2021-08-20 13:26:29 BlockManager: INFO: Found block rdd_11_0 locally 2021-08-20 13:26:29 root: INFO: TaskReport: stage=1, partition=0, attempt=0, peakBytes=393216, peakBytesReadable=384.00 KiB, chunks requested=0, cache hits=0 2021-08-20 13:26:29 root: INFO: RegionPool: FREE: 384.0K allocated (384.0K blocks / 0 chunks), regions.size = 6, 0 current java objects, thread 123: Executor task launch worker for task 0.0 in stage 1.0 (TID 1) 2021-08-20 13:26:29 Executor: INFO: 1 block locks were not released by task 0.0 in stage 1.0 (TID 1) [rdd_11_0] 2021-08-20 13:26:29 Executor: INFO: Finished task 0.0 in stage 1.0 (TID 1). 1053 bytes result sent to driver 2021-08-20 13:26:29 TaskSetManager: INFO: Finished task 0.0 in stage 1.0 (TID 1) in 67 ms on spark-master (executor driver) (1/1) 2021-08-20 13:26:29 TaskSchedulerImpl: INFO: Removed TaskSet 1.0, whose tasks have all completed, from pool 2021-08-20 13:26:29 DAGScheduler: INFO: ResultStage 1 (collect at RVD.scala:763) finished in 0.097 s 2021-08-20 13:26:29 DAGScheduler: INFO: Job 1 is finished. Cancelling potential speculative or zombie tasks for this job 2021-08-20 13:26:29 TaskSchedulerImpl: INFO: Killing all running tasks in stage 1: Stage finished 2021-08-20 13:26:29 DAGScheduler: INFO: Job 1 finished: collect at RVD.scala:763, took 0.114088 s 2021-08-20 13:26:29 root: INFO: decoder cache miss (1 hits, 6 misses, 0.143 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C269FSContainer. 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C269FSContainer. 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C270etypeDecode. 2021-08-20 13:26:29 root: INFO: instruction count: 5: __C270etypeDecode.apply 2021-08-20 13:26:29 root: INFO: instruction count: 198: __C270etypeDecode.__m271DECODE_r_struct_of_r_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryEND_TO_o_struct_of_r_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringEND 2021-08-20 13:26:29 root: INFO: instruction count: 33: __C270etypeDecode.__m272INPLACE_DECODE_r_binary_TO_r_string 2021-08-20 13:26:29 root: INFO: took 2.661s 2021-08-20 13:26:29 root: INFO: optimize optimize: relationalLowerer, after InterpretNonCompilable: before: IR size 2: (GetField rows (Literal Struct{rows:Array[Struct{locus:String,alleles:String,rsid:String,qual:String,filters:String,`info.IMPRECISE`:String,`info.SVTYPE`:String,`info.SVLEN`:String,`info.END`:String,`info.CIPOS`:String,`info.CIEND`:String,`info.CIGAR`:String,`info.MATEID`:String,`info.EVENT`:String,`info.HOMLEN`:String,`info.HOMSEQ`:String,`info.SVINSLEN`:String,`info.SVINSSEQ`:String,`info.LEFT_SVINSSEQ`:String,`info.RIGHT_SVINSSEQ`:String,`info.BND_DEPTH`:String,`info.MATE_BND_DEPTH`:String,`info.JUNCTION_QUAL`:String,vep:String,`vep_proc_id.part_idx`:String,`vep_proc_id.block_idx`:String}],global:Struct{}} )) 2021-08-20 13:26:29 root: INFO: optimize optimize: relationalLowerer, after InterpretNonCompilable: after: IR size 1: (Literal Array[Struct{locus:String,alleles:String,rsid:String,qual:String,filters:String,`info.IMPRECISE`:String,`info.SVTYPE`:String,`info.SVLEN`:String,`info.END`:String,`info.CIPOS`:String,`info.CIEND`:String,`info.CIGAR`:String,`info.MATEID`:String,`info.EVENT`:String,`info.HOMLEN`:String,`info.HOMSEQ`:String,`info.SVINSLEN`:String,`info.SVINSSEQ`:String,`info.LEFT_SVINSSEQ`:String,`info.RIGHT_SVINSSEQ`:String,`info.BND_DEPTH`:String,`info.MATE_BND_DEPTH`:String,`info.JUNCTION_QUAL`:String,vep:String,`vep_proc_id.part_idx`:String,`vep_proc_id.block_idx`:String}] ) 2021-08-20 13:26:29 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 2: (MakeTuple (0) (Literal Array[Struct{locus:String,alleles:String,rsid:String,qual:String,filters:String,`info.IMPRECISE`:String,`info.SVTYPE`:String,`info.SVLEN`:String,`info.END`:String,`info.CIPOS`:String,`info.CIEND`:String,`info.CIGAR`:String,`info.MATEID`:String,`info.EVENT`:String,`info.HOMLEN`:String,`info.HOMSEQ`:String,`info.SVINSLEN`:String,`info.SVINSSEQ`:String,`info.LEFT_SVINSSEQ`:String,`info.RIGHT_SVINSSEQ`:String,`info.BND_DEPTH`:String,`info.MATE_BND_DEPTH`:String,`info.JUNCTION_QUAL`:String,vep:String,`vep_proc_id.part_idx`:String,`vep_proc_id.block_idx`:String}] )) 2021-08-20 13:26:29 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 1: (Literal Tuple[Array[Struct{locus:String,alleles:String,rsid:String,qual:String,filters:String,`info.IMPRECISE`:String,`info.SVTYPE`:String,`info.SVLEN`:String,`info.END`:String,`info.CIPOS`:String,`info.CIEND`:String,`info.CIGAR`:String,`info.MATEID`:String,`info.EVENT`:String,`info.HOMLEN`:String,`info.HOMSEQ`:String,`info.SVINSLEN`:String,`info.SVINSSEQ`:String,`info.LEFT_SVINSSEQ`:String,`info.RIGHT_SVINSSEQ`:String,`info.BND_DEPTH`:String,`info.MATE_BND_DEPTH`:String,`info.JUNCTION_QUAL`:String,vep:String,`vep_proc_id.part_idx`:String,`vep_proc_id.block_idx`:String}]] ) 2021-08-20 13:26:29 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 1: (Literal Tuple[Array[Struct{locus:String,alleles:String,rsid:String,qual:String,filters:String,`info.IMPRECISE`:String,`info.SVTYPE`:String,`info.SVLEN`:String,`info.END`:String,`info.CIPOS`:String,`info.CIEND`:String,`info.CIGAR`:String,`info.MATEID`:String,`info.EVENT`:String,`info.HOMLEN`:String,`info.HOMSEQ`:String,`info.SVINSLEN`:String,`info.SVINSSEQ`:String,`info.LEFT_SVINSSEQ`:String,`info.RIGHT_SVINSSEQ`:String,`info.BND_DEPTH`:String,`info.MATE_BND_DEPTH`:String,`info.JUNCTION_QUAL`:String,vep:String,`vep_proc_id.part_idx`:String,`vep_proc_id.block_idx`:String}]] ) 2021-08-20 13:26:29 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 1: (Literal Tuple[Array[Struct{locus:String,alleles:String,rsid:String,qual:String,filters:String,`info.IMPRECISE`:String,`info.SVTYPE`:String,`info.SVLEN`:String,`info.END`:String,`info.CIPOS`:String,`info.CIEND`:String,`info.CIGAR`:String,`info.MATEID`:String,`info.EVENT`:String,`info.HOMLEN`:String,`info.HOMSEQ`:String,`info.SVINSLEN`:String,`info.SVINSSEQ`:String,`info.LEFT_SVINSSEQ`:String,`info.RIGHT_SVINSSEQ`:String,`info.BND_DEPTH`:String,`info.MATE_BND_DEPTH`:String,`info.JUNCTION_QUAL`:String,vep:String,`vep_proc_id.part_idx`:String,`vep_proc_id.block_idx`:String}]] ) 2021-08-20 13:26:29 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 1: (Literal Tuple[Array[Struct{locus:String,alleles:String,rsid:String,qual:String,filters:String,`info.IMPRECISE`:String,`info.SVTYPE`:String,`info.SVLEN`:String,`info.END`:String,`info.CIPOS`:String,`info.CIEND`:String,`info.CIGAR`:String,`info.MATEID`:String,`info.EVENT`:String,`info.HOMLEN`:String,`info.HOMSEQ`:String,`info.SVINSLEN`:String,`info.SVINSSEQ`:String,`info.LEFT_SVINSSEQ`:String,`info.RIGHT_SVINSSEQ`:String,`info.BND_DEPTH`:String,`info.MATE_BND_DEPTH`:String,`info.JUNCTION_QUAL`:String,vep:String,`vep_proc_id.part_idx`:String,`vep_proc_id.block_idx`:String}]] ) 2021-08-20 13:26:29 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 1: (Literal Tuple[Array[Struct{locus:String,alleles:String,rsid:String,qual:String,filters:String,`info.IMPRECISE`:String,`info.SVTYPE`:String,`info.SVLEN`:String,`info.END`:String,`info.CIPOS`:String,`info.CIEND`:String,`info.CIGAR`:String,`info.MATEID`:String,`info.EVENT`:String,`info.HOMLEN`:String,`info.HOMSEQ`:String,`info.SVINSLEN`:String,`info.SVINSSEQ`:String,`info.LEFT_SVINSSEQ`:String,`info.RIGHT_SVINSSEQ`:String,`info.BND_DEPTH`:String,`info.MATE_BND_DEPTH`:String,`info.JUNCTION_QUAL`:String,vep:String,`vep_proc_id.part_idx`:String,`vep_proc_id.block_idx`:String}]] ) 2021-08-20 13:26:29 root: INFO: encoder cache miss (1 hits, 5 misses, 0.167) 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C284FSContainer. 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C284FSContainer. 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C285etypeEncode. 2021-08-20 13:26:29 root: INFO: instruction count: 5: __C285etypeEncode.apply 2021-08-20 13:26:29 root: INFO: instruction count: 9: __C285etypeEncode.__m286ENCODE_o_tuple_of_r_tuple_of_r_array_of_r_struct_of_r_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringENDENDEND_TO_r_struct_of_r_struct_of_r_array_of_r_struct_of_r_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryENDENDEND 2021-08-20 13:26:29 root: INFO: instruction count: 13: __C285etypeEncode.__m287ENCODE_o_tuple_of_r_array_of_r_struct_of_r_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringENDEND_TO_r_struct_of_r_array_of_r_struct_of_r_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryENDEND 2021-08-20 13:26:29 root: INFO: instruction count: 45: __C285etypeEncode.__m288ENCODE_o_array_of_r_struct_of_r_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringEND_TO_r_array_of_r_struct_of_r_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryEND 2021-08-20 13:26:29 root: INFO: instruction count: 263: __C285etypeEncode.__m289ENCODE_o_struct_of_r_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringEND_TO_r_struct_of_r_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryEND 2021-08-20 13:26:29 root: INFO: instruction count: 18: __C285etypeEncode.__m290ENCODE_o_string_TO_r_binary 2021-08-20 13:26:29 MemoryStore: INFO: Block broadcast_6 stored as values in memory (estimated size 336.0 B, free 26.5 GiB) 2021-08-20 13:26:29 MemoryStore: INFO: Block broadcast_6_piece0 stored as bytes in memory (estimated size 220.0 B, free 26.5 GiB) 2021-08-20 13:26:29 BlockManagerInfo: INFO: Added broadcast_6_piece0 in memory on spark-master:32903 (size: 220.0 B, free: 26.5 GiB) 2021-08-20 13:26:29 SparkContext: INFO: Created broadcast 6 from broadcast at SparkBackend.scala:278 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C273FSContainer. 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C273FSContainer. 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C274Compiled. 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C274Compiled.apply 2021-08-20 13:26:29 root: INFO: instruction count: 9: __C274Compiled.setPartitionIndex 2021-08-20 13:26:29 root: INFO: instruction count: 4: __C274Compiled.addPartitionRegion 2021-08-20 13:26:29 root: INFO: instruction count: 4: __C274Compiled.setPool 2021-08-20 13:26:29 root: INFO: instruction count: 3: __C274Compiled.addFS 2021-08-20 13:26:29 root: INFO: instruction count: 32: __C274Compiled.addLiterals 2021-08-20 13:26:29 root: INFO: instruction count: 23: __C274Compiled.__m279DECODE_r_struct_of_r_struct_of_r_array_of_r_struct_of_r_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryENDENDEND_TO_o_tuple_of_r_tuple_of_r_array_of_r_struct_of_r_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringENDENDEND 2021-08-20 13:26:29 root: INFO: instruction count: 15: __C274Compiled.__m280INPLACE_DECODE_r_struct_of_r_array_of_r_struct_of_r_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryENDEND_TO_r_tuple_of_r_array_of_r_struct_of_r_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringENDEND 2021-08-20 13:26:29 root: INFO: instruction count: 64: __C274Compiled.__m281INPLACE_DECODE_r_array_of_r_struct_of_r_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryEND_TO_r_array_of_r_struct_of_r_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringEND 2021-08-20 13:26:29 root: INFO: instruction count: 190: __C274Compiled.__m282INPLACE_DECODE_r_struct_of_r_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryANDr_binaryEND_TO_r_struct_of_r_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringANDr_stringEND 2021-08-20 13:26:29 root: INFO: instruction count: 33: __C274Compiled.__m283INPLACE_DECODE_r_binary_TO_r_string 2021-08-20 13:26:30 root: INFO: finished execution of query hail_query_1 2021-08-20 13:26:30 root: INFO: RegionPool: FREE: 128.0K allocated (128.0K blocks / 0 chunks), regions.size = 2, 0 current java objects, thread 28: Thread-5 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON total 3.212s self 20.310ms children 3.192s %children 99.37% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR total 194.245ms self 0.017ms children 194.227ms %children 99.99% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/Verify total 0.379ms self 0.379ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation total 193.608ms self 24.227ms children 169.382ms %children 87.49% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize total 169.382ms self 0.183ms children 169.199ms %children 99.89% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 2.310ms self 2.310ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 3.923ms self 3.923ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 74.486ms self 74.486ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 29.608ms self 29.608ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.966ms self 0.966ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 17.532ms self 17.532ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 3.904ms self 3.904ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 1.851ms self 1.851ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 1.177ms self 1.177ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 7.134ms self 7.134ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.504ms self 0.504ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 8.962ms self 8.962ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 2.256ms self 2.256ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 1.401ms self 1.401ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.927ms self 0.927ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 5.936ms self 5.936ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.380ms self 0.380ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 5.941ms self 5.941ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/Verify total 0.240ms self 0.240ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable total 18.041ms self 0.017ms children 18.024ms %children 99.91% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable/Verify total 1.212ms self 1.212ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable/LoweringTransformation total 16.101ms self 16.101ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable/Verify total 0.712ms self 0.712ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable total 214.100ms self 0.031ms children 214.069ms %children 99.99% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/Verify total 0.054ms self 0.054ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation total 213.910ms self 166.541ms children 47.369ms %children 22.14% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize total 47.369ms self 0.144ms children 47.225ms %children 99.70% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/FoldConstants total 1.517ms self 1.517ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ExtractIntervalFilters total 1.669ms self 1.669ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/Simplify total 1.333ms self 1.333ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardLets total 5.800ms self 5.800ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.376ms self 0.376ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/PruneDeadFields total 8.084ms self 8.084ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/FoldConstants total 0.683ms self 0.683ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ExtractIntervalFilters total 1.581ms self 1.581ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/Simplify total 1.517ms self 1.517ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardLets total 5.752ms self 5.752ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.302ms self 0.302ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/PruneDeadFields total 5.857ms self 5.857ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/FoldConstants total 0.873ms self 0.873ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ExtractIntervalFilters total 1.145ms self 1.145ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/Simplify total 0.883ms self 0.883ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardLets total 4.216ms self 4.216ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.278ms self 0.278ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/PruneDeadFields total 5.359ms self 5.359ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/Verify total 0.105ms self 0.105ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable total 2.662s self 0.034ms children 2.662s %children 100.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/Verify total 0.194ms self 0.194ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation total 2.662s self 2.525s children 137.286ms %children 5.16% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR total 3.862ms self 0.017ms children 3.845ms %children 99.56% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.050ms self 0.050ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation total 3.790ms self 2.406ms children 1.384ms %children 36.51% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize total 1.384ms self 0.075ms children 1.309ms %children 94.55% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.370ms self 0.370ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.087ms self 0.087ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.192ms self 0.192ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.235ms self 0.235ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.056ms self 0.056ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.139ms self 0.139ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.048ms self 0.048ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.007ms self 0.007ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.021ms self 0.021ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.099ms self 0.099ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.017ms self 0.017ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.038ms self 0.038ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.005ms self 0.005ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR total 0.735ms self 0.012ms children 0.722ms %children 98.33% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.291ms self 0.291ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/LoweringTransformation total 0.417ms self 0.417ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.015ms self 0.015ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR total 0.879ms self 0.008ms children 0.871ms %children 99.04% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.002ms self 0.002ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation total 0.866ms self 0.630ms children 0.236ms %children 27.29% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize total 0.236ms self 0.023ms children 0.213ms %children 90.18% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.028ms self 0.028ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.011ms self 0.011ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.022ms self 0.022ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 0.094ms self 0.094ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.020ms self 0.020ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 0.039ms self 0.039ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.003ms self 0.003ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs total 2.714ms self 0.009ms children 2.705ms %children 99.66% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.005ms self 0.005ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/LoweringTransformation total 2.466ms self 2.466ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.235ms self 0.235ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs total 0.883ms self 0.009ms children 0.875ms %children 99.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.002ms self 0.002ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation total 0.870ms self 0.640ms children 0.230ms %children 26.44% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize total 0.230ms self 0.016ms children 0.214ms %children 93.12% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.027ms self 0.027ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.011ms self 0.011ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.021ms self 0.021ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 0.097ms self 0.097ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.027ms self 0.027ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 0.031ms self 0.031ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.003ms self 0.003ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/EmitContext.analyze total 3.873ms self 3.873ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR total 11.231ms self 0.023ms children 11.208ms %children 99.79% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.038ms self 0.038ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation total 11.159ms self 3.010ms children 8.149ms %children 73.03% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize total 8.149ms self 0.108ms children 8.040ms %children 98.67% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.346ms self 0.346ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.084ms self 0.084ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.572ms self 0.572ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 1.039ms self 1.039ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.093ms self 0.093ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 2.553ms self 2.553ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.139ms self 0.139ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.068ms self 0.068ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.155ms self 0.155ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.659ms self 0.659ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.060ms self 0.060ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.616ms self 0.616ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.152ms self 0.152ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.063ms self 0.063ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.128ms self 0.128ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.657ms self 0.657ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.055ms self 0.055ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.600ms self 0.600ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.011ms self 0.011ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR total 0.166ms self 0.009ms children 0.158ms %children 94.80% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.035ms self 0.035ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/LoweringTransformation total 0.106ms self 0.106ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.016ms self 0.016ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR total 9.573ms self 0.012ms children 9.561ms %children 99.87% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.016ms self 0.016ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation total 9.535ms self 3.344ms children 6.191ms %children 64.93% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize total 6.191ms self 0.096ms children 6.095ms %children 98.45% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.109ms self 0.109ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.075ms self 0.075ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.150ms self 0.150ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 0.854ms self 0.854ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.063ms self 0.063ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 0.689ms self 0.689ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.131ms self 0.131ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.069ms self 0.069ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.147ms self 0.147ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 1.086ms self 1.086ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.079ms self 0.079ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 0.741ms self 0.741ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.117ms self 0.117ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.076ms self 0.076ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.140ms self 0.140ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 0.781ms self 0.781ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.059ms self 0.059ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 0.728ms self 0.728ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.010ms self 0.010ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs total 5.814ms self 0.009ms children 5.804ms %children 99.84% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.019ms self 0.019ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/LoweringTransformation total 5.765ms self 5.765ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.020ms self 0.020ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs total 6.127ms self 0.010ms children 6.117ms %children 99.84% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.005ms self 0.005ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation total 6.104ms self 1.856ms children 4.248ms %children 69.59% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize total 4.248ms self 0.066ms children 4.182ms %children 98.44% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.070ms self 0.070ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.064ms self 0.064ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.140ms self 0.140ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 0.601ms self 0.601ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.048ms self 0.048ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 0.477ms self 0.477ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.096ms self 0.096ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.053ms self 0.053ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.120ms self 0.120ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 0.648ms self 0.648ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.047ms self 0.047ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 0.465ms self 0.465ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.083ms self 0.083ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.055ms self 0.055ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.112ms self 0.112ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 0.577ms self 0.577ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.047ms self 0.047ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 0.479ms self 0.479ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.008ms self 0.008ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/EmitContext.analyze total 2.357ms self 2.357ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR total 28.001ms self 0.016ms children 27.985ms %children 99.94% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.089ms self 0.089ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation total 27.854ms self 4.778ms children 23.076ms %children 82.85% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize total 23.076ms self 0.156ms children 22.920ms %children 99.32% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.969ms self 0.969ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.271ms self 0.271ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.621ms self 0.621ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 2.247ms self 2.247ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.172ms self 0.172ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 4.752ms self 4.752ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.355ms self 0.355ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.267ms self 0.267ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.585ms self 0.585ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 3.140ms self 3.140ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.167ms self 0.167ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 2.947ms self 2.947ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.349ms self 0.349ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.269ms self 0.269ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.573ms self 0.573ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 2.013ms self 2.013ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.174ms self 0.174ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 3.049ms self 3.049ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.043ms self 0.043ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR total 0.384ms self 0.006ms children 0.378ms %children 98.35% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.090ms self 0.090ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/LoweringTransformation total 0.193ms self 0.193ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.094ms self 0.094ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR total 23.676ms self 0.010ms children 23.666ms %children 99.96% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.023ms self 0.023ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation total 23.614ms self 4.162ms children 19.452ms %children 82.37% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize total 19.452ms self 0.070ms children 19.382ms %children 99.64% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.282ms self 0.282ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.285ms self 0.285ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.625ms self 0.625ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 3.119ms self 3.119ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.186ms self 0.186ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 2.807ms self 2.807ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.344ms self 0.344ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.276ms self 0.276ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.569ms self 0.569ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 2.200ms self 2.200ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.155ms self 0.155ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 2.780ms self 2.780ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.496ms self 0.496ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.265ms self 0.265ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.579ms self 0.579ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 1.858ms self 1.858ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.153ms self 0.153ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 2.402ms self 2.402ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.029ms self 0.029ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs total 7.156ms self 0.009ms children 7.147ms %children 99.87% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.112ms self 0.112ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/LoweringTransformation total 6.942ms self 6.942ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.093ms self 0.093ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs total 24.700ms self 0.012ms children 24.689ms %children 99.95% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.022ms self 0.022ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation total 24.628ms self 4.410ms children 20.217ms %children 82.09% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize total 20.217ms self 0.062ms children 20.155ms %children 99.69% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.298ms self 0.298ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.284ms self 0.284ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.647ms self 0.647ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 3.164ms self 3.164ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.187ms self 0.187ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 2.480ms self 2.480ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.354ms self 0.354ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.255ms self 0.255ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.579ms self 0.579ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 3.687ms self 3.687ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.147ms self 0.147ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 2.416ms self 2.416ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.335ms self 0.335ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.261ms self 0.261ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.575ms self 0.575ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 1.733ms self 1.733ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.140ms self 0.140ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 2.613ms self 2.613ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.039ms self 0.039ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/EmitContext.analyze total 5.152ms self 5.152ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/Verify total 0.048ms self 0.048ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable total 1.765ms self 0.009ms children 1.756ms %children 99.51% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation total 1.748ms self 0.610ms children 1.138ms %children 65.11% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize total 1.138ms self 0.059ms children 1.078ms %children 94.77% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/FoldConstants total 0.476ms self 0.476ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.014ms self 0.014ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/Simplify total 0.042ms self 0.042ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardLets total 0.204ms self 0.204ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.038ms self 0.038ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/PruneDeadFields total 0.085ms self 0.085ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/FoldConstants total 0.043ms self 0.043ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.008ms self 0.008ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/Simplify total 0.024ms self 0.024ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardLets total 0.097ms self 0.097ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.017ms self 0.017ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/PruneDeadFields total 0.031ms self 0.031ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile total 98.504ms self 32.381ms children 66.123ms %children 67.13% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR total 63.932ms self 0.013ms children 63.919ms %children 99.98% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/Verify total 0.005ms self 0.005ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation total 63.902ms self 62.495ms children 1.407ms %children 2.20% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize total 1.407ms self 0.034ms children 1.373ms %children 97.60% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.967ms self 0.967ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.013ms self 0.013ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.041ms self 0.041ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.133ms self 0.133ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.026ms self 0.026ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.054ms self 0.054ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.033ms self 0.033ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.004ms self 0.004ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.011ms self 0.011ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.060ms self 0.060ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.010ms self 0.010ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.020ms self 0.020ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/Verify total 0.011ms self 0.011ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR total 0.030ms self 0.005ms children 0.025ms %children 83.83% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR/Verify total 0.008ms self 0.008ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR/LoweringTransformation total 0.014ms self 0.014ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR/Verify total 0.003ms self 0.003ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR total 0.454ms self 0.004ms children 0.449ms %children 99.03% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/Verify total 0.001ms self 0.001ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation total 0.446ms self 0.232ms children 0.213ms %children 47.87% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize total 0.213ms self 0.012ms children 0.201ms %children 94.34% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.033ms self 0.033ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.010ms self 0.010ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.022ms self 0.022ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 0.086ms self 0.086ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.018ms self 0.018ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 0.033ms self 0.033ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/Verify total 0.002ms self 0.002ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs total 0.936ms self 0.006ms children 0.930ms %children 99.36% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs/Verify total 0.005ms self 0.005ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs/LoweringTransformation total 0.917ms self 0.917ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs/Verify total 0.008ms self 0.008ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs total 0.505ms self 0.006ms children 0.499ms %children 98.78% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.001ms self 0.001ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation total 0.495ms self 0.301ms children 0.194ms %children 39.15% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize total 0.194ms self 0.016ms children 0.177ms %children 91.59% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.028ms self 0.028ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.009ms self 0.009ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.019ms self 0.019ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 0.077ms self 0.077ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.014ms self 0.014ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 0.030ms self 0.030ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.003ms self 0.003ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/Compile/EmitContext.analyze total 0.266ms self 0.266ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/InitializeCompiledFunction total 1.753ms self 1.753ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/RunCompiledFunction total 0.008ms self 0.008ms children 0.000ms %children 0.00% 2021-08-20 13:26:30 root: INFO: timing SparkBackend.executeJSON/convertRegionValueToAnnotation total 0.638ms self 0.638ms children 0.000ms %children 0.00% 2021-08-20 13:32:03 AbstractConnector: INFO: Stopped Spark@56c3ea63{HTTP/1.1, (http/1.1)}{10.1.1.3:4040} 2021-08-20 13:32:03 SparkUI: INFO: Stopped Spark web UI at http://spark-master:4040 2021-08-20 13:32:03 MapOutputTrackerMasterEndpoint: INFO: MapOutputTrackerMasterEndpoint stopped! 2021-08-20 13:32:03 MemoryStore: INFO: MemoryStore cleared 2021-08-20 13:32:03 BlockManager: INFO: BlockManager stopped 2021-08-20 13:32:03 BlockManagerMaster: INFO: BlockManagerMaster stopped 2021-08-20 13:32:03 OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: INFO: OutputCommitCoordinator stopped! 2021-08-20 13:32:03 SparkContext: INFO: Successfully stopped SparkContext 2021-08-20 13:32:03 SparkContext: INFO: SparkContext already stopped.