2021-09-20 12:07:50 Hail: INFO: Running Hail version 0.2.61-3c86d3ba497a 2021-09-20 12:07:50 CoarseGrainedSchedulerBackend$DriverEndpoint: INFO: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.0.3.1:59756) with ID 0 2021-09-20 12:07:50 SharedState: INFO: loading hive config file: file:/cluster/spark/conf/hive-site.xml 2021-09-20 12:07:50 SharedState: INFO: Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('/cluster/spark/spark-warehouse'). 2021-09-20 12:07:50 SharedState: INFO: Warehouse path is '/cluster/spark/spark-warehouse'. 2021-09-20 12:07:50 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@60d1f03f{/SQL,null,AVAILABLE,@Spark} 2021-09-20 12:07:50 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@52ffdbeb{/SQL/json,null,AVAILABLE,@Spark} 2021-09-20 12:07:50 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@30a897a{/SQL/execution,null,AVAILABLE,@Spark} 2021-09-20 12:07:50 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@27616538{/SQL/execution/json,null,AVAILABLE,@Spark} 2021-09-20 12:07:50 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@b9c9bb2{/static/sql,null,AVAILABLE,@Spark} 2021-09-20 12:07:50 BlockManagerMasterEndpoint: INFO: Registering block manager ip-10-60-141-91.eu-west-2.compute.internal:44000 with 11.5 GB RAM, BlockManagerId(0, ip-10-60-141-91.eu-west-2.compute.internal, 44000, None) 2021-09-20 12:07:51 StateStoreCoordinatorRef: INFO: Registered StateStoreCoordinator endpoint 2021-09-20 12:07:51 SparkSession$Builder: WARN: Using an existing SparkSession; some configuration may not take effect. 2021-09-20 12:07:51 root: INFO: RegionPool: initialized for thread 24: Thread-4 2021-09-20 12:07:51 MemoryStore: INFO: Block broadcast_0 stored as values in memory (estimated size 1699.1 KB, free 11.5 GB) 2021-09-20 12:07:51 MemoryStore: INFO: Block broadcast_0_piece0 stored as bytes in memory (estimated size 123.7 KB, free 11.5 GB) 2021-09-20 12:07:51 BlockManagerInfo: INFO: Added broadcast_0_piece0 in memory on ip-10-60-47-217.eu-west-2.compute.internal:43001 (size: 123.7 KB, free: 11.5 GB) 2021-09-20 12:07:51 SparkContext: INFO: Created broadcast 0 from broadcast at SparkBackend.scala:250 2021-09-20 12:07:51 root: INFO: timing SparkBackend.parse_matrix_ir total 607.393ms self 607.393ms children 0.000ms %children 0.00% 2021-09-20 12:07:51 root: INFO: timing SparkBackend.parse_value_ir total 3.607ms self 3.607ms children 0.000ms %children 0.00% 2021-09-20 12:07:51 root: INFO: starting execution of query hail_query_1 of initial size 3 2021-09-20 12:07:52 root: INFO: optimize optimize: relationalLowerer, initial IR: before: IR size 3: (TableCount (MatrixColsTable (MatrixRead None False False "{\"path\":\"file:/opt/notebooks/GIPR.mt\",\"name\":\"MatrixNativeReader\"}"))) 2021-09-20 12:07:52 root: INFO: optimize optimize: relationalLowerer, initial IR: after: IR size 1: (I64 200643) 2021-09-20 12:07:52 root: INFO: optimize optimize: relationalLowerer, after LowerMatrixToTable: before: IR size 1: (I64 200643) 2021-09-20 12:07:52 root: INFO: optimize optimize: relationalLowerer, after LowerMatrixToTable: after: IR size 1: (I64 200643) 2021-09-20 12:07:52 root: INFO: optimize optimize: relationalLowerer, after InterpretNonCompilable: before: IR size 1: (I64 200643) 2021-09-20 12:07:52 root: INFO: optimize optimize: relationalLowerer, after InterpretNonCompilable: after: IR size 1: (I64 200643) 2021-09-20 12:07:52 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 2: (MakeTuple (0) (I64 200643)) 2021-09-20 12:07:52 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 1: (Literal Tuple[Int64] ) 2021-09-20 12:07:52 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 1: (Literal Tuple[Int64] ) 2021-09-20 12:07:52 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 1: (Literal Tuple[Int64] ) 2021-09-20 12:07:52 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 1: (Literal Tuple[Int64] ) 2021-09-20 12:07:52 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 1: (Literal Tuple[Int64] ) 2021-09-20 12:07:52 root: INFO: encoder cache miss (0 hits, 1 misses, 0.000) 2021-09-20 12:07:52 root: INFO: instruction count: 3: __C8etypeEncode. 2021-09-20 12:07:52 root: INFO: instruction count: 5: __C8etypeEncode.apply 2021-09-20 12:07:52 root: INFO: instruction count: 9: __C8etypeEncode.__m9ENCODE_r_tuple_of_r_tuple_of_r_int64ENDEND_TO_r_struct_of_r_struct_of_r_int64ENDEND 2021-09-20 12:07:52 root: INFO: instruction count: 13: __C8etypeEncode.__m10ENCODE_r_tuple_of_r_int64END_TO_r_struct_of_r_int64END 2021-09-20 12:07:52 root: INFO: instruction count: 4: __C8etypeEncode.__m11ENCODE_r_int64_TO_r_int64 2021-09-20 12:07:52 MemoryStore: INFO: Block broadcast_1 stored as values in memory (estimated size 80.0 B, free 11.5 GB) 2021-09-20 12:07:52 MemoryStore: INFO: Block broadcast_1_piece0 stored as bytes in memory (estimated size 67.0 B, free 11.5 GB) 2021-09-20 12:07:52 BlockManagerInfo: INFO: Added broadcast_1_piece0 in memory on ip-10-60-47-217.eu-west-2.compute.internal:43001 (size: 67.0 B, free: 11.5 GB) 2021-09-20 12:07:52 SparkContext: INFO: Created broadcast 1 from broadcast at SparkBackend.scala:250 2021-09-20 12:07:52 root: INFO: instruction count: 3: __C1Compiled. 2021-09-20 12:07:52 root: INFO: instruction count: 3: __C1Compiled.apply 2021-09-20 12:07:52 root: INFO: instruction count: 9: __C1Compiled.setPartitionIndex 2021-09-20 12:07:52 root: INFO: instruction count: 4: __C1Compiled.addPartitionRegion 2021-09-20 12:07:52 root: INFO: instruction count: 23: __C1Compiled.__m5DECODE_r_struct_of_r_struct_of_r_int64ENDEND_TO_r_tuple_of_r_tuple_of_r_int64ENDEND 2021-09-20 12:07:52 root: INFO: instruction count: 15: __C1Compiled.__m6INPLACE_DECODE_r_struct_of_r_int64END_TO_r_tuple_of_r_int64END 2021-09-20 12:07:52 root: INFO: instruction count: 8: __C1Compiled.__m7INPLACE_DECODE_r_int64_TO_r_int64 2021-09-20 12:07:52 root: INFO: instruction count: 32: __C1Compiled.addLiterals 2021-09-20 12:07:52 root: INFO: finished execution of query hail_query_1 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON total 309.075ms self 49.257ms children 259.818ms %children 84.06% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR total 69.146ms self 0.427ms children 68.719ms %children 99.38% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/Verify total 0.258ms self 0.258ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation total 68.445ms self 39.179ms children 29.266ms %children 42.76% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize total 29.266ms self 0.911ms children 28.355ms %children 96.89% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 2.439ms self 2.439ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 3.155ms self 3.155ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 8.979ms self 8.979ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 4.641ms self 4.641ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.890ms self 0.890ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 8.058ms self 8.058ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.048ms self 0.048ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.005ms self 0.005ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.013ms self 0.013ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.091ms self 0.091ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.012ms self 0.012ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.023ms self 0.023ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/Verify total 0.017ms self 0.017ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable total 4.810ms self 0.011ms children 4.799ms %children 99.76% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable/Verify total 0.002ms self 0.002ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable/LoweringTransformation total 4.781ms self 4.781ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable/Verify total 0.015ms self 0.015ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable total 0.619ms self 0.010ms children 0.609ms %children 98.35% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/Verify total 0.002ms self 0.002ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation total 0.602ms self 0.356ms children 0.246ms %children 40.83% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize total 0.246ms self 0.021ms children 0.224ms %children 91.26% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/FoldConstants total 0.053ms self 0.053ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.006ms self 0.006ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/Simplify total 0.017ms self 0.017ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardLets total 0.099ms self 0.099ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.018ms self 0.018ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/PruneDeadFields total 0.032ms self 0.032ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable total 0.973ms self 0.015ms children 0.958ms %children 98.47% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation total 0.833ms self 0.833ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/Verify total 0.121ms self 0.121ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable total 0.489ms self 0.009ms children 0.481ms %children 98.24% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/Verify total 0.002ms self 0.002ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation total 0.475ms self 0.299ms children 0.176ms %children 37.02% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize total 0.176ms self 0.016ms children 0.160ms %children 90.96% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/FoldConstants total 0.035ms self 0.035ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.005ms self 0.005ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/Simplify total 0.013ms self 0.013ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardLets total 0.075ms self 0.075ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.011ms self 0.011ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/PruneDeadFields total 0.021ms self 0.021ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile total 174.944ms self 152.859ms children 22.084ms %children 12.62% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR total 12.864ms self 0.011ms children 12.853ms %children 99.91% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/Verify total 0.007ms self 0.007ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation total 12.841ms self 6.162ms children 6.679ms %children 52.02% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize total 6.679ms self 0.045ms children 6.634ms %children 99.32% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 6.343ms self 6.343ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.015ms self 0.015ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.018ms self 0.018ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.085ms self 0.085ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.014ms self 0.014ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.034ms self 0.034ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.031ms self 0.031ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.004ms self 0.004ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.010ms self 0.010ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.051ms self 0.051ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.010ms self 0.010ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.020ms self 0.020ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/Verify total 0.005ms self 0.005ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR total 0.193ms self 0.007ms children 0.187ms %children 96.63% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR/Verify total 0.007ms self 0.007ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR/LoweringTransformation total 0.172ms self 0.172ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR/Verify total 0.009ms self 0.009ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR total 0.461ms self 0.007ms children 0.453ms %children 98.45% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/Verify total 0.001ms self 0.001ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation total 0.449ms self 0.275ms children 0.174ms %children 38.68% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize total 0.174ms self 0.017ms children 0.157ms %children 90.26% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.030ms self 0.030ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.007ms self 0.007ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.015ms self 0.015ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 0.069ms self 0.069ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.012ms self 0.012ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 0.024ms self 0.024ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/Verify total 0.003ms self 0.003ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs total 8.047ms self 0.012ms children 8.036ms %children 99.86% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs/Verify total 0.006ms self 0.006ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs/LoweringTransformation total 7.896ms self 7.896ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs/Verify total 0.134ms self 0.134ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs total 0.519ms self 0.008ms children 0.512ms %children 98.54% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.002ms self 0.002ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation total 0.507ms self 0.314ms children 0.192ms %children 37.94% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize total 0.192ms self 0.018ms children 0.175ms %children 90.79% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.037ms self 0.037ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.007ms self 0.007ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.018ms self 0.018ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 0.077ms self 0.077ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.012ms self 0.012ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 0.024ms self 0.024ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.003ms self 0.003ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/InitializeCompiledFunction total 7.254ms self 7.254ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/RunCompiledFunction total 0.010ms self 0.010ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/convertRegionValueToAnnotation total 1.574ms self 1.574ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.parse_value_ir total 10.574ms self 10.574ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: starting execution of query hail_query_2 of initial size 28 2021-09-20 12:07:52 root: INFO: optimize optimize: relationalLowerer, initial IR: before: IR size 28: (ToArray (StreamMap __uid_4 (ToStream False (GetField rows (TableCollect (TableHead 0 (TableMapRows (MatrixColsTable (MatrixMapCols () (MatrixMapGlobals (MatrixMapCols None (MatrixRead None False False "{\"path\":\"file:/opt/notebooks/GIPR.mt\",\"name\":\"MatrixNativeReader\"}") (InsertFields (SelectFields (s) (SelectFields (s) (Ref sa))) None)) (InsertFields (SelectFields () (SelectFields () (Ref global))) None)) (SelectFields (s) (Ref sa)))) (InsertFields (SelectFields (s) (SelectFields (s) (Ref row))) None)))))) (GetField s (Ref __uid_4)))) 2021-09-20 12:07:52 root: INFO: optimize optimize: relationalLowerer, initial IR: after: IR size 12: (ToArray (StreamMap __iruid_2 (ToStream False (GetField rows (TableCollect (MatrixColsTable (MatrixMapCols () (MatrixColsHead 0 (MatrixRead Matrix{global:Struct{},col_key:[s],col:Struct{s:String},row_key:[[locus]],row:Struct{locus:Locus(GRCh38)},entry:Struct{}} False False "{\"path\":\"file:/opt/notebooks/GIPR.mt\",\"name\":\"MatrixNativeReader\"}")) (Ref sa)))))) (GetField s (Ref __iruid_2)))) 2021-09-20 12:07:52 root: INFO: optimize optimize: relationalLowerer, after LowerMatrixToTable: before: IR size 59: (ToArray (StreamMap __iruid_2 (ToStream False (GetField rows (TableCollect (TableKeyBy () False (TableParallelize None (Let __cols_and_globals (TableGetGlobals (TableMapGlobals (TableMapRows (TableMapGlobals (TableMapGlobals (TableRead Table{global:Struct{},key:[locus],row:Struct{locus:Locus(GRCh38),`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{}]}} False "{\"name\":\"TableNativeZippedReader\",\"pathLeft\":\"file:/opt/notebooks/GIPR.mt/rows\",\"pathRight\":\"file:/opt/notebooks/GIPR.mt/entries\",\"specLeft\":{\"name\":\"TableSpec\",\"params\":{\"file_version\":66816,\"hail_version\":\"0.2.61-3c86d3ba497a\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus],row:Struct{locus:Locus(GRCh38),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AF:Array[Float64],AQ:Array[Int32],AC:Array[Int32],AN:Int32}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[828]}}},\"globalsSpec\":{\"_key\":[],\"_codecSpec\":{\"_eType\":{\"fields\":[],\"required\":true},\"_vType\":{\"fields\":[]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_partFiles\":[\"part-0\"],\"_jRangeBounds\":[{\"start\":{},\"end\":{},\"includeStart\":true,\"includeEnd\":true}],\"_attrs\":{}},\"rowsSpec\":{\"_key\":[\"locus\"],\"_codecSpec\":{\"_eType\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"fields\":[{\"name\":\"contig\",\"typ\":{},\"index\":0},{\"name\":\"position\",\"typ\":{},\"index\":1}],\"required\":true},\"index\":0},{\"name\":\"alleles\",\"typ\":{\"elementType\":{},\"required\":true},\"index\":1},{\"name\":\"rsid\",\"typ\":{},\"index\":2},{\"name\":\"qual\",\"typ\":{},\"index\":3},{\"name\":\"filters\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":4},{\"name\":\"info\",\"typ\":{\"fields\":[{\"name\":\"AF\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":0},{\"name\":\"AQ\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":1},{\"name\":\"AC\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":2},{\"name\":\"AN\",\"typ\":{},\"index\":3}],\"required\":true},\"index\":5}],\"required\":true},\"_vType\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0},{\"name\":\"alleles\",\"typ\":{\"elementType\":{}},\"index\":1},{\"name\":\"rsid\",\"typ\":{},\"index\":2},{\"name\":\"qual\",\"typ\":{},\"index\":3},{\"name\":\"filters\",\"typ\":{\"elementType\":{}},\"index\":4},{\"name\":\"info\",\"typ\":{\"fields\":[{\"name\":\"AF\",\"typ\":{\"elementType\":{}},\"index\":0},{\"name\":\"AQ\",\"typ\":{\"elementType\":{}},\"index\":1},{\"name\":\"AC\",\"typ\":{\"elementType\":{}},\"index\":2},{\"name\":\"AN\",\"typ\":{},\"index\":3}]},\"index\":5}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_indexSpec\":{\"_relPath\":\"../../index\",\"_leafCodec\":{\"_eType\":{\"fields\":[{\"name\":\"first_idx\",\"typ\":{},\"index\":0},{\"name\":\"keys\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"fields\":[{\"name\":\"contig\",\"typ\":{},\"index\":0},{\"name\":\"position\",\"typ\":{},\"index\":1}],\"required\":true},\"index\":0}],\"required\":true},\"index\":0},{\"name\":\"offset\",\"typ\":{},\"index\":1},{\"name\":\"annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}],\"required\":true},\"index\":2}],\"required\":true},\"required\":true},\"index\":1}],\"required\":false},\"_vType\":{\"fields\":[{\"name\":\"first_idx\",\"typ\":{},\"index\":0},{\"name\":\"keys\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"index\":0},{\"name\":\"offset\",\"typ\":{},\"index\":1},{\"name\":\"annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]},\"index\":2}]}},\"index\":1}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_internalNodeCodec\":{\"_eType\":{\"fields\":[{\"name\":\"children\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"index_file_offset\",\"typ\":{},\"index\":0},{\"name\":\"first_idx\",\"typ\":{},\"index\":1},{\"name\":\"first_key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"fields\":[{\"name\":\"contig\",\"typ\":{},\"index\":0},{\"name\":\"position\",\"typ\":{},\"index\":1}],\"required\":true},\"index\":0}],\"required\":true},\"index\":2},{\"name\":\"first_record_offset\",\"typ\":{},\"index\":3},{\"name\":\"first_annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}],\"required\":true},\"index\":4}],\"required\":true},\"required\":true},\"index\":0}],\"required\":false},\"_vType\":{\"fields\":[{\"name\":\"children\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"index_file_offset\",\"typ\":{},\"index\":0},{\"name\":\"first_idx\",\"typ\":{},\"index\":1},{\"name\":\"first_key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"index\":2},{\"name\":\"first_record_offset\",\"typ\":{},\"index\":3},{\"name\":\"first_annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]},\"index\":4}]}},\"index\":0}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_keyType\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"_annotationType\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]}},\"_partFiles\":[\"part-0-2-0-0-de4e832a-2b5f-07d0-d290-a2be64649dc3\"],\"_jRangeBounds\":[{\"start\":{\"locus\":{\"contig\":\"chr19\",\"position\":45668221}},\"end\":{\"locus\":{\"contig\":\"chr19\",\"position\":45683722}},\"includeStart\":true,\"includeEnd\":true}],\"_attrs\":{}}},\"specRight\":{\"name\":\"TableSpec\",\"params\":{\"file_version\":66816,\"hail_version\":\"0.2.61-3c86d3ba497a\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[],row:Struct{`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,RNC:Array[String],DP:Int32,AD:Array[Int32],GQ:Int32,PL:Array[Int32]}]}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[828]}}},\"globalsSpec\":{\"_key\":[],\"_codecSpec\":{\"_eType\":{\"fields\":[],\"required\":true},\"_vType\":{\"fields\":[]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_partFiles\":[\"part-0\"],\"_jRangeBounds\":[{\"start\":{},\"end\":{},\"includeStart\":true,\"includeEnd\":true}],\"_attrs\":{}},\"rowsSpec\":{\"_key\":[],\"_codecSpec\":{\"_eType\":{\"fields\":[{\"name\":\"the entries! [877f12a8827e18f61222c6c8c5fb04a8]\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"GT\",\"typ\":{},\"index\":0},{\"name\":\"RNC\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":1},{\"name\":\"DP\",\"typ\":{},\"index\":2},{\"name\":\"AD\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":3},{\"name\":\"GQ\",\"typ\":{},\"index\":4},{\"name\":\"PL\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":5}],\"required\":true},\"required\":true},\"index\":0}],\"required\":true},\"_vType\":{\"fields\":[{\"name\":\"the entries! [877f12a8827e18f61222c6c8c5fb04a8]\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"GT\",\"typ\":{},\"index\":0},{\"name\":\"RNC\",\"typ\":{\"elementType\":{}},\"index\":1},{\"name\":\"DP\",\"typ\":{},\"index\":2},{\"name\":\"AD\",\"typ\":{\"elementType\":{}},\"index\":3},{\"name\":\"GQ\",\"typ\":{},\"index\":4},{\"name\":\"PL\",\"typ\":{\"elementType\":{}},\"index\":5}]}},\"index\":0}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_indexSpec\":{\"_relPath\":\"../../index\",\"_leafCodec\":{\"_eType\":{\"fields\":[{\"name\":\"first_idx\",\"typ\":{},\"index\":0},{\"name\":\"keys\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"fields\":[{\"name\":\"contig\",\"typ\":{},\"index\":0},{\"name\":\"position\",\"typ\":{},\"index\":1}],\"required\":true},\"index\":0}],\"required\":true},\"index\":0},{\"name\":\"offset\",\"typ\":{},\"index\":1},{\"name\":\"annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}],\"required\":true},\"index\":2}],\"required\":true},\"required\":true},\"index\":1}],\"required\":false},\"_vType\":{\"fields\":[{\"name\":\"first_idx\",\"typ\":{},\"index\":0},{\"name\":\"keys\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"index\":0},{\"name\":\"offset\",\"typ\":{},\"index\":1},{\"name\":\"annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]},\"index\":2}]}},\"index\":1}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_internalNodeCodec\":{\"_eType\":{\"fields\":[{\"name\":\"children\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"index_file_offset\",\"typ\":{},\"index\":0},{\"name\":\"first_idx\",\"typ\":{},\"index\":1},{\"name\":\"first_key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"fields\":[{\"name\":\"contig\",\"typ\":{},\"index\":0},{\"name\":\"position\",\"typ\":{},\"index\":1}],\"required\":true},\"index\":0}],\"required\":true},\"index\":2},{\"name\":\"first_record_offset\",\"typ\":{},\"index\":3},{\"name\":\"first_annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}],\"required\":true},\"index\":4}],\"required\":true},\"required\":true},\"index\":0}],\"required\":false},\"_vType\":{\"fields\":[{\"name\":\"children\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"index_file_offset\",\"typ\":{},\"index\":0},{\"name\":\"first_idx\",\"typ\":{},\"index\":1},{\"name\":\"first_key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"index\":2},{\"name\":\"first_record_offset\",\"typ\":{},\"index\":3},{\"name\":\"first_annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]},\"index\":4}]}},\"index\":0}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_keyType\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"_annotationType\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]},\"_offsetField\":\"entries_offset\"},\"_partFiles\":[\"part-0-2-0-0-de4e832a-2b5f-07d0-d290-a2be64649dc3\"],\"_jRangeBounds\":[{\"start\":{},\"end\":{},\"includeStart\":true,\"includeEnd\":true}],\"_attrs\":{}}}}") (InsertFields (Ref global) None (__cols (ToArray (ReadPartition Struct{s:String} "{\"name\":\"PartitionNativeReader\",\"spec\":{\"name\":\"TypedCodecSpec\",\"_eType\":\"+EBaseStruct{s:+EBinary}\",\"_vType\":\"Struct{s:String}\",\"_bufferSpec\":{\"name\":\"LEB128BufferSpec\",\"child\":{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"LZ4HCBlockBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}}}}}" (Str "file:/opt/...")))))) (InsertFields (Ref global) None (__cols (ApplyIR sliceLeft () Array[Struct{s:String}] (GetField __cols (Ref global)) (I32 0))))) (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ApplyIR sliceLeft () Array[Struct{}] (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (I32 0))))) (InsertFields (Ref global) None (__cols (ToArray (StreamMap __iruid_9 (ToStream False (ToArray (StreamRange False (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)))) (Let __cols_array (GetField __cols (Ref global)) (Let sa (ArrayRef (Ref __cols_array) (Ref __iruid_9) (Str "")) (Ref sa))))))))) (MakeStruct (rows (GetField __cols (Ref __cols_and_globals))) (global (SelectFields () (Ref __cols_and_globals)))))))))) (GetField s (Ref __iruid_2)))) 2021-09-20 12:07:52 root: INFO: Prune: MakeStruct: eliminating field 'global' 2021-09-20 12:07:52 root: INFO: optimize optimize: relationalLowerer, after LowerMatrixToTable: after: IR size 24: (ToArray (StreamMap __iruid_79 (ToStream False (GetField rows (Let __iruid_80 (ApplyIR sliceLeft () Array[Struct{s:String}] (ToArray (ReadPartition Struct{s:String} "{\"name\":\"PartitionNativeReader\",\"spec\":{\"name\":\"TypedCodecSpec\",\"_eType\":\"+EBaseStruct{s:+EBinary}\",\"_vType\":\"Struct{s:String}\",\"_bufferSpec\":{\"name\":\"LEB128BufferSpec\",\"child\":{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"LZ4HCBlockBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}}}}}" (Str "file:/opt/..."))) (I32 0)) (MakeStruct (rows (ToArray (StreamMap __iruid_81 (StreamRange False (I32 0) (ArrayLen (Ref __iruid_80)) (I32 1)) (ArrayRef (Ref __iruid_80) (Ref __iruid_81) (Str ""))))))))) (GetField s (Ref __iruid_79)))) 2021-09-20 12:07:52 root: INFO: optimize optimize: relationalLowerer, after InterpretNonCompilable: before: IR size 24: (ToArray (StreamMap __iruid_79 (ToStream False (GetField rows (Let __iruid_80 (ApplyIR sliceLeft () Array[Struct{s:String}] (ToArray (ReadPartition Struct{s:String} "{\"name\":\"PartitionNativeReader\",\"spec\":{\"name\":\"TypedCodecSpec\",\"_eType\":\"+EBaseStruct{s:+EBinary}\",\"_vType\":\"Struct{s:String}\",\"_bufferSpec\":{\"name\":\"LEB128BufferSpec\",\"child\":{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"LZ4HCBlockBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}}}}}" (Str "file:/opt/..."))) (I32 0)) (MakeStruct (rows (ToArray (StreamMap __iruid_81 (StreamRange False (I32 0) (ArrayLen (Ref __iruid_80)) (I32 1)) (ArrayRef (Ref __iruid_80) (Ref __iruid_81) (Str ""))))))))) (GetField s (Ref __iruid_79)))) 2021-09-20 12:07:52 root: INFO: optimize optimize: relationalLowerer, after InterpretNonCompilable: after: IR size 24: (ToArray (StreamMap __iruid_124 (ToStream False (GetField rows (Let __iruid_125 (ApplyIR sliceLeft () Array[Struct{s:String}] (ToArray (ReadPartition Struct{s:String} "{\"name\":\"PartitionNativeReader\",\"spec\":{\"name\":\"TypedCodecSpec\",\"_eType\":\"+EBaseStruct{s:+EBinary}\",\"_vType\":\"Struct{s:String}\",\"_bufferSpec\":{\"name\":\"LEB128BufferSpec\",\"child\":{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"LZ4HCBlockBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}}}}}" (Str "file:/opt/..."))) (I32 0)) (MakeStruct (rows (ToArray (StreamMap __iruid_126 (StreamRange False (I32 0) (ArrayLen (Ref __iruid_125)) (I32 1)) (ArrayRef (Ref __iruid_125) (Ref __iruid_126) (Str ""))))))))) (GetField s (Ref __iruid_124)))) 2021-09-20 12:07:52 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 25: (MakeTuple (0) (ToArray (StreamMap __iruid_124 (ToStream False (GetField rows (Let __iruid_125 (ApplyIR sliceLeft () Array[Struct{s:String}] (ToArray (ReadPartition Struct{s:String} "{\"name\":\"PartitionNativeReader\",\"spec\":{\"name\":\"TypedCodecSpec\",\"_eType\":\"+EBaseStruct{s:+EBinary}\",\"_vType\":\"Struct{s:String}\",\"_bufferSpec\":{\"name\":\"LEB128BufferSpec\",\"child\":{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"LZ4HCBlockBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}}}}}" (Str "file:/opt/..."))) (I32 0)) (MakeStruct (rows (ToArray (StreamMap __iruid_126 (StreamRange False (I32 0) (ArrayLen (Ref __iruid_125)) (I32 1)) (ArrayRef (Ref __iruid_125) (Ref __iruid_126) (Str ""))))))))) (GetField s (Ref __iruid_124))))) 2021-09-20 12:07:52 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 25: (MakeTuple (0) (ToArray (StreamMap __iruid_169 (ToStream False (GetField rows (Let __iruid_170 (ApplyIR sliceLeft () Array[Struct{s:String}] (ToArray (ReadPartition Struct{s:String} "{\"name\":\"PartitionNativeReader\",\"spec\":{\"name\":\"TypedCodecSpec\",\"_eType\":\"+EBaseStruct{s:+EBinary}\",\"_vType\":\"Struct{s:String}\",\"_bufferSpec\":{\"name\":\"LEB128BufferSpec\",\"child\":{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"LZ4HCBlockBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}}}}}" (Str "file:/opt/..."))) (I32 0)) (MakeStruct (rows (ToArray (StreamMap __iruid_171 (StreamRange False (I32 0) (ArrayLen (Ref __iruid_170)) (I32 1)) (ArrayRef (Ref __iruid_170) (Ref __iruid_171) (Str ""))))))))) (GetField s (Ref __iruid_169))))) 2021-09-20 12:07:52 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 55: (MakeTuple (0) (ToArray (StreamMap __iruid_169 (ToStream False (GetField rows (Let __iruid_170 (Let __iruid_178 (ToArray (ReadPartition Struct{s:String} "{\"name\":\"PartitionNativeReader\",\"spec\":{\"name\":\"TypedCodecSpec\",\"_eType\":\"+EBaseStruct{s:+EBinary}\",\"_vType\":\"Struct{s:String}\",\"_bufferSpec\":{\"name\":\"LEB128BufferSpec\",\"child\":{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"LZ4HCBlockBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}}}}}" (Str "file:/opt/..."))) (Let __iruid_179 (I32 0) (If (IsNA (Ref __iruid_178)) (Ref __iruid_178) (ToArray (StreamMap __iruid_180 (StreamRange False (I32 0) (If (ApplyComparisonOp LT (Ref __iruid_179) (I32 0)) (ApplyBinaryPrimOp Add (ArrayLen (Ref __iruid_178)) (Ref __iruid_179)) (If (ApplyComparisonOp LT (Ref __iruid_179) (ArrayLen (Ref __iruid_178))) (Ref __iruid_179) (ArrayLen (Ref __iruid_178)))) (I32 1)) (ArrayRef (Ref __iruid_178) (Ref __iruid_180) (Str ""))))))) (MakeStruct (rows (ToArray (StreamMap __iruid_171 (StreamRange False (I32 0) (ArrayLen (Ref __iruid_170)) (I32 1)) (ArrayRef (Ref __iruid_170) (Ref __iruid_171) (Str ""))))))))) (GetField s (Ref __iruid_169))))) 2021-09-20 12:07:52 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 45: (MakeTuple (0) (ToArray (StreamMap __iruid_192 (ToStream False (GetField rows (Let __iruid_193 (ToArray (ReadPartition Struct{s:String} "{\"name\":\"PartitionNativeReader\",\"spec\":{\"name\":\"TypedCodecSpec\",\"_eType\":\"+EBaseStruct{s:+EBinary}\",\"_vType\":\"Struct{s:String}\",\"_bufferSpec\":{\"name\":\"LEB128BufferSpec\",\"child\":{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"LZ4HCBlockBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}}}}}" (Str "file:/opt/..."))) (Let __iruid_194 (If (IsNA (Ref __iruid_193)) (Ref __iruid_193) (ToArray (StreamMap __iruid_195 (StreamRange False (I32 0) (If (ApplyComparisonOp LT (I32 0) (ArrayLen (Ref __iruid_193))) (I32 0) (ArrayLen (Ref __iruid_193))) (I32 1)) (ArrayRef (Ref __iruid_193) (Ref __iruid_195) (Str ""))))) (MakeStruct (rows (ToArray (StreamMap __iruid_196 (StreamRange False (I32 0) (ArrayLen (Ref __iruid_194)) (I32 1)) (ArrayRef (Ref __iruid_194) (Ref __iruid_196) (Str "")))))))))) (GetField s (Ref __iruid_192))))) 2021-09-20 12:07:52 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 45: (MakeTuple (0) (ToArray (StreamMap __iruid_192 (ToStream False (GetField rows (Let __iruid_193 (ToArray (ReadPartition Struct{s:String} "{\"name\":\"PartitionNativeReader\",\"spec\":{\"name\":\"TypedCodecSpec\",\"_eType\":\"+EBaseStruct{s:+EBinary}\",\"_vType\":\"Struct{s:String}\",\"_bufferSpec\":{\"name\":\"LEB128BufferSpec\",\"child\":{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"LZ4HCBlockBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}}}}}" (Str "file:/opt/..."))) (Let __iruid_194 (If (IsNA (Ref __iruid_193)) (Ref __iruid_193) (ToArray (StreamMap __iruid_195 (StreamRange False (I32 0) (If (ApplyComparisonOp LT (I32 0) (ArrayLen (Ref __iruid_193))) (I32 0) (ArrayLen (Ref __iruid_193))) (I32 1)) (ArrayRef (Ref __iruid_193) (Ref __iruid_195) (Str ""))))) (MakeStruct (rows (ToArray (StreamMap __iruid_196 (StreamRange False (I32 0) (ArrayLen (Ref __iruid_194)) (I32 1)) (ArrayRef (Ref __iruid_194) (Ref __iruid_196) (Str "")))))))))) (GetField s (Ref __iruid_192))))) 2021-09-20 12:07:52 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 45: (MakeTuple (0) (ToArray (StreamMap __iruid_207 (ToStream False (GetField rows (Let __iruid_208 (ToArray (ReadPartition Struct{s:String} "{\"name\":\"PartitionNativeReader\",\"spec\":{\"name\":\"TypedCodecSpec\",\"_eType\":\"+EBaseStruct{s:+EBinary}\",\"_vType\":\"Struct{s:String}\",\"_bufferSpec\":{\"name\":\"LEB128BufferSpec\",\"child\":{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"LZ4HCBlockBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}}}}}" (Str "file:/opt/..."))) (Let __iruid_209 (If (IsNA (Ref __iruid_208)) (Ref __iruid_208) (ToArray (StreamMap __iruid_210 (StreamRange False (I32 0) (If (ApplyComparisonOp LT (I32 0) (ArrayLen (Ref __iruid_208))) (I32 0) (ArrayLen (Ref __iruid_208))) (I32 1)) (ArrayRef (Ref __iruid_208) (Ref __iruid_210) (Str ""))))) (MakeStruct (rows (ToArray (StreamMap __iruid_211 (StreamRange False (I32 0) (ArrayLen (Ref __iruid_209)) (I32 1)) (ArrayRef (Ref __iruid_209) (Ref __iruid_211) (Str "")))))))))) (GetField s (Ref __iruid_207))))) 2021-09-20 12:07:52 root: INFO: encoder cache miss (0 hits, 2 misses, 0.000) 2021-09-20 12:07:52 root: INFO: instruction count: 3: __C82etypeEncode. 2021-09-20 12:07:52 root: INFO: instruction count: 5: __C82etypeEncode.apply 2021-09-20 12:07:52 root: INFO: instruction count: 13: __C82etypeEncode.__m83ENCODE_r_tuple_of_r_stringEND_TO_r_struct_of_r_binaryEND 2021-09-20 12:07:52 root: INFO: instruction count: 16: __C82etypeEncode.__m84ENCODE_r_binary_TO_r_binary 2021-09-20 12:07:52 MemoryStore: INFO: Block broadcast_2 stored as values in memory (estimated size 128.0 B, free 11.5 GB) 2021-09-20 12:07:52 MemoryStore: INFO: Block broadcast_2_piece0 stored as bytes in memory (estimated size 113.0 B, free 11.5 GB) 2021-09-20 12:07:52 BlockManagerInfo: INFO: Added broadcast_2_piece0 in memory on ip-10-60-47-217.eu-west-2.compute.internal:43001 (size: 113.0 B, free: 11.5 GB) 2021-09-20 12:07:52 SparkContext: INFO: Created broadcast 2 from broadcast at SparkBackend.scala:250 2021-09-20 12:07:52 root: INFO: instruction count: 3: __C12Compiled. 2021-09-20 12:07:52 root: INFO: instruction count: 743: __C12Compiled.apply 2021-09-20 12:07:52 root: INFO: instruction count: 23: __C12Compiled.__m19DECODE_r_struct_of_r_binaryEND_TO_r_tuple_of_r_stringEND 2021-09-20 12:07:52 root: INFO: instruction count: 31: __C12Compiled.__m20INPLACE_DECODE_r_binary_TO_r_binary 2021-09-20 12:07:52 root: INFO: instruction count: 12: __C12Compiled.__m27setup_jab 2021-09-20 12:07:52 root: INFO: instruction count: 10: __C12Compiled.__m30addIRIntermediate 2021-09-20 12:07:52 root: INFO: instruction count: 18: __C12Compiled.__m40cord 2021-09-20 12:07:52 root: INFO: instruction count: 5: __C12Compiled.__m66addIRIntermediate 2021-09-20 12:07:52 root: INFO: instruction count: 5: __C12Compiled.__m77addIRIntermediate 2021-09-20 12:07:52 root: INFO: instruction count: 5: __C12Compiled.__m78addIRIntermediate 2021-09-20 12:07:52 root: INFO: instruction count: 9: __C12Compiled.setPartitionIndex 2021-09-20 12:07:52 root: INFO: instruction count: 4: __C12Compiled.addPartitionRegion 2021-09-20 12:07:52 root: INFO: instruction count: 23: __C12Compiled.__m81DECODE_r_struct_of_r_binaryEND_TO_r_tuple_of_r_stringEND 2021-09-20 12:07:52 root: INFO: instruction count: 36: __C12Compiled.addLiterals 2021-09-20 12:07:52 root: INFO: instruction count: 487: __C12Compiled.applyregion57_84 2021-09-20 12:07:52 root: INFO: instruction count: 3: __C85applySpills. 2021-09-20 12:07:52 root: INFO: instruction count: 3: __C29FSContainer. 2021-09-20 12:07:52 root: INFO: instruction count: 28: __C29FSContainer.init_filesystem 2021-09-20 12:07:52 root: INFO: instruction count: 3: __C29FSContainer. 2021-09-20 12:07:52 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 24: Thread-4 2021-09-20 12:07:52 root: INFO: RegionPool: REPORT_THRESHOLD: 512.0K allocated (512.0K blocks / 0 chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 24: Thread-4 2021-09-20 12:07:52 root: INFO: RegionPool: REPORT_THRESHOLD: 1.0M allocated (1.0M blocks / 0 chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 24: Thread-4 2021-09-20 12:07:52 root: INFO: RegionPool: REPORT_THRESHOLD: 2.0M allocated (2.0M blocks / 0 chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 24: Thread-4 2021-09-20 12:07:52 root: INFO: RegionPool: REPORT_THRESHOLD: 4.0M allocated (4.0M blocks / 0 chunks), regions.size = 2, 0 current java objects, 0 max java objects, thread 24: Thread-4 2021-09-20 12:07:52 root: INFO: finished execution of query hail_query_2 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON total 556.750ms self 5.553ms children 551.197ms %children 99.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR total 28.546ms self 0.015ms children 28.531ms %children 99.95% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/Verify total 0.040ms self 0.040ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation total 28.470ms self 8.206ms children 20.264ms %children 71.18% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize total 20.264ms self 0.151ms children 20.112ms %children 99.25% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.382ms self 0.382ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.764ms self 0.764ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 2.369ms self 2.369ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 3.022ms self 3.022ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.073ms self 0.073ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 8.729ms self 8.729ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.143ms self 0.143ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.047ms self 0.047ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.145ms self 0.145ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.997ms self 0.997ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.061ms self 0.061ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.861ms self 0.861ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.141ms self 0.141ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.056ms self 0.056ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.130ms self 0.130ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 1.272ms self 1.272ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.057ms self 0.057ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.861ms self 0.861ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/Verify total 0.021ms self 0.021ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable total 103.266ms self 0.020ms children 103.246ms %children 99.98% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable/Verify total 0.017ms self 0.017ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable/LoweringTransformation total 102.849ms self 102.849ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable/Verify total 0.380ms self 0.380ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable total 103.084ms self 0.023ms children 103.061ms %children 99.98% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/Verify total 0.068ms self 0.068ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation total 102.940ms self 71.373ms children 31.567ms %children 30.67% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize total 31.567ms self 0.105ms children 31.462ms %children 99.67% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/FoldConstants total 2.867ms self 2.867ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ExtractIntervalFilters total 1.285ms self 1.285ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/Simplify total 16.161ms self 16.161ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardLets total 2.101ms self 2.101ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.290ms self 0.290ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/PruneDeadFields total 2.827ms self 2.827ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/FoldConstants total 0.306ms self 0.306ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.084ms self 0.084ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/Simplify total 0.497ms self 0.497ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardLets total 1.321ms self 1.321ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.087ms self 0.087ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/PruneDeadFields total 0.892ms self 0.892ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/FoldConstants total 0.274ms self 0.274ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.067ms self 0.067ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/Simplify total 0.325ms self 0.325ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardLets total 1.128ms self 1.128ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.087ms self 0.087ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/PruneDeadFields total 0.865ms self 0.865ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/Verify total 0.054ms self 0.054ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable total 2.217ms self 0.008ms children 2.209ms %children 99.62% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/Verify total 0.026ms self 0.026ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation total 0.067ms self 0.067ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/Verify total 2.116ms self 2.116ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable total 13.552ms self 0.009ms children 13.543ms %children 99.93% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/Verify total 0.025ms self 0.025ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation total 13.468ms self 2.694ms children 10.775ms %children 80.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize total 10.775ms self 0.066ms children 10.709ms %children 99.39% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/FoldConstants total 0.384ms self 0.384ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.073ms self 0.073ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/Simplify total 0.394ms self 0.394ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardLets total 2.573ms self 2.573ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.105ms self 0.105ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/PruneDeadFields total 0.915ms self 0.915ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/FoldConstants total 0.255ms self 0.255ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.069ms self 0.069ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/Simplify total 0.316ms self 0.316ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardLets total 1.441ms self 1.441ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.094ms self 0.094ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/PruneDeadFields total 1.219ms self 1.219ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/FoldConstants total 0.239ms self 0.239ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.084ms self 0.084ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/Simplify total 0.284ms self 0.284ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardLets total 1.236ms self 1.236ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.077ms self 0.077ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/PruneDeadFields total 0.952ms self 0.952ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/Verify total 0.049ms self 0.049ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile total 211.428ms self 147.403ms children 64.025ms %children 30.28% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR total 16.448ms self 0.011ms children 16.437ms %children 99.93% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/Verify total 0.011ms self 0.011ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation total 16.412ms self 2.387ms children 14.025ms %children 85.46% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize total 14.025ms self 0.089ms children 13.936ms %children 99.36% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.288ms self 0.288ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.072ms self 0.072ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.333ms self 0.333ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 2.771ms self 2.771ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.098ms self 0.098ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 2.180ms self 2.180ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.297ms self 0.297ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.072ms self 0.072ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.232ms self 0.232ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 2.956ms self 2.956ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.098ms self 0.098ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 1.258ms self 1.258ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.219ms self 0.219ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.073ms self 0.073ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.254ms self 0.254ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 1.333ms self 1.333ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.107ms self 0.107ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 1.296ms self 1.296ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/Verify total 0.014ms self 0.014ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR total 0.419ms self 0.009ms children 0.410ms %children 97.84% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR/Verify total 0.045ms self 0.045ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR/LoweringTransformation total 0.243ms self 0.243ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR/Verify total 0.122ms self 0.122ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR total 21.952ms self 0.010ms children 21.942ms %children 99.96% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/Verify total 0.019ms self 0.019ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation total 21.906ms self 2.913ms children 18.993ms %children 86.70% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize total 18.993ms self 0.080ms children 18.914ms %children 99.58% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.269ms self 0.269ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.121ms self 0.121ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 1.233ms self 1.233ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 3.894ms self 3.894ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.208ms self 0.208ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 1.582ms self 1.582ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 1.547ms self 1.547ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.138ms self 0.138ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.373ms self 0.373ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 4.784ms self 4.784ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.186ms self 0.186ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 0.994ms self 0.994ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.225ms self 0.225ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.150ms self 0.150ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.242ms self 0.242ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 1.894ms self 1.894ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.122ms self 0.122ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 0.954ms self 0.954ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/Verify total 0.018ms self 0.018ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs total 12.894ms self 0.011ms children 12.883ms %children 99.91% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs/Verify total 0.056ms self 0.056ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs/LoweringTransformation total 12.782ms self 12.782ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs/Verify total 0.045ms self 0.045ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs total 12.312ms self 0.016ms children 12.296ms %children 99.87% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.011ms self 0.011ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation total 12.271ms self 2.337ms children 9.935ms %children 80.96% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize total 9.935ms self 0.174ms children 9.761ms %children 98.25% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.150ms self 0.150ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.099ms self 0.099ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.271ms self 0.271ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 1.718ms self 1.718ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.163ms self 0.163ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 1.067ms self 1.067ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.257ms self 0.257ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.097ms self 0.097ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.180ms self 0.180ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 2.270ms self 2.270ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.112ms self 0.112ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 0.776ms self 0.776ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.185ms self 0.185ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.074ms self 0.074ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.180ms self 0.180ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 1.248ms self 1.248ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.120ms self 0.120ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 0.792ms self 0.792ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.014ms self 0.014ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/InitializeCompiledFunction total 2.072ms self 2.072ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/RunCompiledFunction total 85.776ms self 85.776ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.executeJSON/convertRegionValueToAnnotation total 1.256ms self 1.256ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: timing SparkBackend.parse_value_ir total 6.524ms self 6.524ms children 0.000ms %children 0.00% 2021-09-20 12:07:52 root: INFO: starting execution of query hail_query_3 of initial size 29 2021-09-20 12:07:52 root: INFO: optimize optimize: relationalLowerer, initial IR: before: IR size 29: (GetField rows (TableCollect (TableHead 11 (TableMapRows (TableMapRows (TableOrderBy () (TableMapRows (TableOrderBy (Alocus) (CastMatrixToTable "entries" "cols" (MatrixRead None False False "{\"path\":\"file:/opt/notebooks/GIPR.mt\",\"name\":\"MatrixNativeReader\"}"))) (InsertFields (SelectFields () (SelectFields (locus alleles rsid qual filters info entries) (Ref row))) None (locus (GetField locus (Ref row)))))) (InsertFields (SelectFields () (SelectFields (locus) (Ref row))) None (locus (GetField locus (Ref row))))) (InsertFields (SelectFields () (SelectFields (locus) (Ref row))) None (locus (ApplySpecial showStr () String (GetField locus (Ref row))))))))) 2021-09-20 12:07:52 root: INFO: optimize optimize: relationalLowerer, initial IR: after: IR size 22: (GetField rows (TableCollect (TableMapRows (TableHead 11 (TableOrderBy () (TableOrderBy (Alocus) (TableMapRows (CastMatrixToTable "entries" "cols" (MatrixRead Matrix{global:Struct{},col_key:[s],col:Struct{s:String},row_key:[[locus]],row:Struct{locus:Locus(GRCh38)},entry:Struct{}} False False "{\"path\":\"file:/opt/notebooks/GIPR.mt\",\"name\":\"MatrixNativeReader\"}")) (If (IsNA (Ref row)) (NA Struct{locus:Locus(GRCh38)}) (MakeStruct (locus (GetField locus (Ref row))))))))) (InsertFields (SelectFields () (Ref row)) None (locus (ApplySpecial showStr () String (GetField locus (Ref row)))))))) 2021-09-20 12:07:52 root: INFO: optimize optimize: relationalLowerer, after LowerMatrixToTable: before: IR size 31: (GetField rows (TableCollect (TableMapRows (TableHead 11 (TableOrderBy () (TableOrderBy (Alocus) (TableMapRows (TableRename ( "the entries! [877f12a8827e18f61222c6c8c5fb04a8]") ("entries") ("__cols") ("cols") (TableMapRows (TableMapGlobals (TableRead Table{global:Struct{},key:[locus],row:Struct{locus:Locus(GRCh38),`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{}]}} False "{\"name\":\"TableNativeZippedReader\",\"pathLeft\":\"file:/opt/notebooks/GIPR.mt/rows\",\"pathRight\":\"file:/opt/notebooks/GIPR.mt/entries\",\"specLeft\":{\"name\":\"TableSpec\",\"params\":{\"file_version\":66816,\"hail_version\":\"0.2.61-3c86d3ba497a\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus],row:Struct{locus:Locus(GRCh38),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AF:Array[Float64],AQ:Array[Int32],AC:Array[Int32],AN:Int32}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[828]}}},\"globalsSpec\":{\"_key\":[],\"_codecSpec\":{\"_eType\":{\"fields\":[],\"required\":true},\"_vType\":{\"fields\":[]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_partFiles\":[\"part-0\"],\"_jRangeBounds\":[{\"start\":{},\"end\":{},\"includeStart\":true,\"includeEnd\":true}],\"_attrs\":{}},\"rowsSpec\":{\"_key\":[\"locus\"],\"_codecSpec\":{\"_eType\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"fields\":[{\"name\":\"contig\",\"typ\":{},\"index\":0},{\"name\":\"position\",\"typ\":{},\"index\":1}],\"required\":true},\"index\":0},{\"name\":\"alleles\",\"typ\":{\"elementType\":{},\"required\":true},\"index\":1},{\"name\":\"rsid\",\"typ\":{},\"index\":2},{\"name\":\"qual\",\"typ\":{},\"index\":3},{\"name\":\"filters\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":4},{\"name\":\"info\",\"typ\":{\"fields\":[{\"name\":\"AF\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":0},{\"name\":\"AQ\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":1},{\"name\":\"AC\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":2},{\"name\":\"AN\",\"typ\":{},\"index\":3}],\"required\":true},\"index\":5}],\"required\":true},\"_vType\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0},{\"name\":\"alleles\",\"typ\":{\"elementType\":{}},\"index\":1},{\"name\":\"rsid\",\"typ\":{},\"index\":2},{\"name\":\"qual\",\"typ\":{},\"index\":3},{\"name\":\"filters\",\"typ\":{\"elementType\":{}},\"index\":4},{\"name\":\"info\",\"typ\":{\"fields\":[{\"name\":\"AF\",\"typ\":{\"elementType\":{}},\"index\":0},{\"name\":\"AQ\",\"typ\":{\"elementType\":{}},\"index\":1},{\"name\":\"AC\",\"typ\":{\"elementType\":{}},\"index\":2},{\"name\":\"AN\",\"typ\":{},\"index\":3}]},\"index\":5}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_indexSpec\":{\"_relPath\":\"../../index\",\"_leafCodec\":{\"_eType\":{\"fields\":[{\"name\":\"first_idx\",\"typ\":{},\"index\":0},{\"name\":\"keys\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"fields\":[{\"name\":\"contig\",\"typ\":{},\"index\":0},{\"name\":\"position\",\"typ\":{},\"index\":1}],\"required\":true},\"index\":0}],\"required\":true},\"index\":0},{\"name\":\"offset\",\"typ\":{},\"index\":1},{\"name\":\"annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}],\"required\":true},\"index\":2}],\"required\":true},\"required\":true},\"index\":1}],\"required\":false},\"_vType\":{\"fields\":[{\"name\":\"first_idx\",\"typ\":{},\"index\":0},{\"name\":\"keys\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"index\":0},{\"name\":\"offset\",\"typ\":{},\"index\":1},{\"name\":\"annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]},\"index\":2}]}},\"index\":1}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_internalNodeCodec\":{\"_eType\":{\"fields\":[{\"name\":\"children\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"index_file_offset\",\"typ\":{},\"index\":0},{\"name\":\"first_idx\",\"typ\":{},\"index\":1},{\"name\":\"first_key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"fields\":[{\"name\":\"contig\",\"typ\":{},\"index\":0},{\"name\":\"position\",\"typ\":{},\"index\":1}],\"required\":true},\"index\":0}],\"required\":true},\"index\":2},{\"name\":\"first_record_offset\",\"typ\":{},\"index\":3},{\"name\":\"first_annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}],\"required\":true},\"index\":4}],\"required\":true},\"required\":true},\"index\":0}],\"required\":false},\"_vType\":{\"fields\":[{\"name\":\"children\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"index_file_offset\",\"typ\":{},\"index\":0},{\"name\":\"first_idx\",\"typ\":{},\"index\":1},{\"name\":\"first_key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"index\":2},{\"name\":\"first_record_offset\",\"typ\":{},\"index\":3},{\"name\":\"first_annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]},\"index\":4}]}},\"index\":0}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_keyType\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"_annotationType\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]}},\"_partFiles\":[\"part-0-2-0-0-de4e832a-2b5f-07d0-d290-a2be64649dc3\"],\"_jRangeBounds\":[{\"start\":{\"locus\":{\"contig\":\"chr19\",\"position\":45668221}},\"end\":{\"locus\":{\"contig\":\"chr19\",\"position\":45683722}},\"includeStart\":true,\"includeEnd\":true}],\"_attrs\":{}}},\"specRight\":{\"name\":\"TableSpec\",\"params\":{\"file_version\":66816,\"hail_version\":\"0.2.61-3c86d3ba497a\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[],row:Struct{`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,RNC:Array[String],DP:Int32,AD:Array[Int32],GQ:Int32,PL:Array[Int32]}]}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[828]}}},\"globalsSpec\":{\"_key\":[],\"_codecSpec\":{\"_eType\":{\"fields\":[],\"required\":true},\"_vType\":{\"fields\":[]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_partFiles\":[\"part-0\"],\"_jRangeBounds\":[{\"start\":{},\"end\":{},\"includeStart\":true,\"includeEnd\":true}],\"_attrs\":{}},\"rowsSpec\":{\"_key\":[],\"_codecSpec\":{\"_eType\":{\"fields\":[{\"name\":\"the entries! [877f12a8827e18f61222c6c8c5fb04a8]\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"GT\",\"typ\":{},\"index\":0},{\"name\":\"RNC\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":1},{\"name\":\"DP\",\"typ\":{},\"index\":2},{\"name\":\"AD\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":3},{\"name\":\"GQ\",\"typ\":{},\"index\":4},{\"name\":\"PL\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":5}],\"required\":true},\"required\":true},\"index\":0}],\"required\":true},\"_vType\":{\"fields\":[{\"name\":\"the entries! [877f12a8827e18f61222c6c8c5fb04a8]\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"GT\",\"typ\":{},\"index\":0},{\"name\":\"RNC\",\"typ\":{\"elementType\":{}},\"index\":1},{\"name\":\"DP\",\"typ\":{},\"index\":2},{\"name\":\"AD\",\"typ\":{\"elementType\":{}},\"index\":3},{\"name\":\"GQ\",\"typ\":{},\"index\":4},{\"name\":\"PL\",\"typ\":{\"elementType\":{}},\"index\":5}]}},\"index\":0}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_indexSpec\":{\"_relPath\":\"../../index\",\"_leafCodec\":{\"_eType\":{\"fields\":[{\"name\":\"first_idx\",\"typ\":{},\"index\":0},{\"name\":\"keys\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"fields\":[{\"name\":\"contig\",\"typ\":{},\"index\":0},{\"name\":\"position\",\"typ\":{},\"index\":1}],\"required\":true},\"index\":0}],\"required\":true},\"index\":0},{\"name\":\"offset\",\"typ\":{},\"index\":1},{\"name\":\"annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}],\"required\":true},\"index\":2}],\"required\":true},\"required\":true},\"index\":1}],\"required\":false},\"_vType\":{\"fields\":[{\"name\":\"first_idx\",\"typ\":{},\"index\":0},{\"name\":\"keys\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"index\":0},{\"name\":\"offset\",\"typ\":{},\"index\":1},{\"name\":\"annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]},\"index\":2}]}},\"index\":1}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_internalNodeCodec\":{\"_eType\":{\"fields\":[{\"name\":\"children\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"index_file_offset\",\"typ\":{},\"index\":0},{\"name\":\"first_idx\",\"typ\":{},\"index\":1},{\"name\":\"first_key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"fields\":[{\"name\":\"contig\",\"typ\":{},\"index\":0},{\"name\":\"position\",\"typ\":{},\"index\":1}],\"required\":true},\"index\":0}],\"required\":true},\"index\":2},{\"name\":\"first_record_offset\",\"typ\":{},\"index\":3},{\"name\":\"first_annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}],\"required\":true},\"index\":4}],\"required\":true},\"required\":true},\"index\":0}],\"required\":false},\"_vType\":{\"fields\":[{\"name\":\"children\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"index_file_offset\",\"typ\":{},\"index\":0},{\"name\":\"first_idx\",\"typ\":{},\"index\":1},{\"name\":\"first_key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"index\":2},{\"name\":\"first_record_offset\",\"typ\":{},\"index\":3},{\"name\":\"first_annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]},\"index\":4}]}},\"index\":0}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_keyType\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"_annotationType\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]},\"_offsetField\":\"entries_offset\"},\"_partFiles\":[\"part-0-2-0-0-de4e832a-2b5f-07d0-d290-a2be64649dc3\"],\"_jRangeBounds\":[{\"start\":{},\"end\":{},\"includeStart\":true,\"includeEnd\":true}],\"_attrs\":{}}}}") (InsertFields (Ref global) None (__cols (ToArray (ReadPartition Struct{s:String} "{\"name\":\"PartitionNativeReader\",\"spec\":{\"name\":\"TypedCodecSpec\",\"_eType\":\"+EBaseStruct{s:+EBinary}\",\"_vType\":\"Struct{s:String}\",\"_bufferSpec\":{\"name\":\"LEB128BufferSpec\",\"child\":{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"LZ4HCBlockBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}}}}}" (Str "file:/opt/...")))))) (SelectFields (locus `the entries! [877f12a8827e18f61222c6c8c5fb04a8]`) (Ref row)))) (If (IsNA (Ref row)) (NA Struct{locus:Locus(GRCh38)}) (MakeStruct (locus (GetField locus (Ref row))))))))) (InsertFields (SelectFields () (Ref row)) None (locus (ApplySpecial showStr () String (GetField locus (Ref row)))))))) 2021-09-20 12:07:52 root: INFO: Prune: InsertFields: eliminating field '__cols' 2021-09-20 12:07:53 root: INFO: optimize optimize: relationalLowerer, after LowerMatrixToTable: after: IR size 21: (GetField rows (TableCollect (TableMapRows (TableHead 11 (TableOrderBy () (TableOrderBy (Alocus) (TableMapRows (TableRead Table{global:Struct{},key:[locus],row:Struct{locus:Locus(GRCh38)}} False "{\"name\":\"TableNativeZippedReader\",\"pathLeft\":\"file:/opt/notebooks/GIPR.mt/rows\",\"pathRight\":\"file:/opt/notebooks/GIPR.mt/entries\",\"specLeft\":{\"name\":\"TableSpec\",\"params\":{\"file_version\":66816,\"hail_version\":\"0.2.61-3c86d3ba497a\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus],row:Struct{locus:Locus(GRCh38),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AF:Array[Float64],AQ:Array[Int32],AC:Array[Int32],AN:Int32}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[828]}}},\"globalsSpec\":{\"_key\":[],\"_codecSpec\":{\"_eType\":{\"fields\":[],\"required\":true},\"_vType\":{\"fields\":[]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_partFiles\":[\"part-0\"],\"_jRangeBounds\":[{\"start\":{},\"end\":{},\"includeStart\":true,\"includeEnd\":true}],\"_attrs\":{}},\"rowsSpec\":{\"_key\":[\"locus\"],\"_codecSpec\":{\"_eType\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"fields\":[{\"name\":\"contig\",\"typ\":{},\"index\":0},{\"name\":\"position\",\"typ\":{},\"index\":1}],\"required\":true},\"index\":0},{\"name\":\"alleles\",\"typ\":{\"elementType\":{},\"required\":true},\"index\":1},{\"name\":\"rsid\",\"typ\":{},\"index\":2},{\"name\":\"qual\",\"typ\":{},\"index\":3},{\"name\":\"filters\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":4},{\"name\":\"info\",\"typ\":{\"fields\":[{\"name\":\"AF\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":0},{\"name\":\"AQ\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":1},{\"name\":\"AC\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":2},{\"name\":\"AN\",\"typ\":{},\"index\":3}],\"required\":true},\"index\":5}],\"required\":true},\"_vType\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0},{\"name\":\"alleles\",\"typ\":{\"elementType\":{}},\"index\":1},{\"name\":\"rsid\",\"typ\":{},\"index\":2},{\"name\":\"qual\",\"typ\":{},\"index\":3},{\"name\":\"filters\",\"typ\":{\"elementType\":{}},\"index\":4},{\"name\":\"info\",\"typ\":{\"fields\":[{\"name\":\"AF\",\"typ\":{\"elementType\":{}},\"index\":0},{\"name\":\"AQ\",\"typ\":{\"elementType\":{}},\"index\":1},{\"name\":\"AC\",\"typ\":{\"elementType\":{}},\"index\":2},{\"name\":\"AN\",\"typ\":{},\"index\":3}]},\"index\":5}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_indexSpec\":{\"_relPath\":\"../../index\",\"_leafCodec\":{\"_eType\":{\"fields\":[{\"name\":\"first_idx\",\"typ\":{},\"index\":0},{\"name\":\"keys\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"fields\":[{\"name\":\"contig\",\"typ\":{},\"index\":0},{\"name\":\"position\",\"typ\":{},\"index\":1}],\"required\":true},\"index\":0}],\"required\":true},\"index\":0},{\"name\":\"offset\",\"typ\":{},\"index\":1},{\"name\":\"annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}],\"required\":true},\"index\":2}],\"required\":true},\"required\":true},\"index\":1}],\"required\":false},\"_vType\":{\"fields\":[{\"name\":\"first_idx\",\"typ\":{},\"index\":0},{\"name\":\"keys\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"index\":0},{\"name\":\"offset\",\"typ\":{},\"index\":1},{\"name\":\"annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]},\"index\":2}]}},\"index\":1}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_internalNodeCodec\":{\"_eType\":{\"fields\":[{\"name\":\"children\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"index_file_offset\",\"typ\":{},\"index\":0},{\"name\":\"first_idx\",\"typ\":{},\"index\":1},{\"name\":\"first_key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"fields\":[{\"name\":\"contig\",\"typ\":{},\"index\":0},{\"name\":\"position\",\"typ\":{},\"index\":1}],\"required\":true},\"index\":0}],\"required\":true},\"index\":2},{\"name\":\"first_record_offset\",\"typ\":{},\"index\":3},{\"name\":\"first_annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}],\"required\":true},\"index\":4}],\"required\":true},\"required\":true},\"index\":0}],\"required\":false},\"_vType\":{\"fields\":[{\"name\":\"children\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"index_file_offset\",\"typ\":{},\"index\":0},{\"name\":\"first_idx\",\"typ\":{},\"index\":1},{\"name\":\"first_key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"index\":2},{\"name\":\"first_record_offset\",\"typ\":{},\"index\":3},{\"name\":\"first_annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]},\"index\":4}]}},\"index\":0}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_keyType\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"_annotationType\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]}},\"_partFiles\":[\"part-0-2-0-0-de4e832a-2b5f-07d0-d290-a2be64649dc3\"],\"_jRangeBounds\":[{\"start\":{\"locus\":{\"contig\":\"chr19\",\"position\":45668221}},\"end\":{\"locus\":{\"contig\":\"chr19\",\"position\":45683722}},\"includeStart\":true,\"includeEnd\":true}],\"_attrs\":{}}},\"specRight\":{\"name\":\"TableSpec\",\"params\":{\"file_version\":66816,\"hail_version\":\"0.2.61-3c86d3ba497a\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[],row:Struct{`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,RNC:Array[String],DP:Int32,AD:Array[Int32],GQ:Int32,PL:Array[Int32]}]}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[828]}}},\"globalsSpec\":{\"_key\":[],\"_codecSpec\":{\"_eType\":{\"fields\":[],\"required\":true},\"_vType\":{\"fields\":[]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_partFiles\":[\"part-0\"],\"_jRangeBounds\":[{\"start\":{},\"end\":{},\"includeStart\":true,\"includeEnd\":true}],\"_attrs\":{}},\"rowsSpec\":{\"_key\":[],\"_codecSpec\":{\"_eType\":{\"fields\":[{\"name\":\"the entries! [877f12a8827e18f61222c6c8c5fb04a8]\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"GT\",\"typ\":{},\"index\":0},{\"name\":\"RNC\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":1},{\"name\":\"DP\",\"typ\":{},\"index\":2},{\"name\":\"AD\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":3},{\"name\":\"GQ\",\"typ\":{},\"index\":4},{\"name\":\"PL\",\"typ\":{\"elementType\":{},\"required\":false},\"index\":5}],\"required\":true},\"required\":true},\"index\":0}],\"required\":true},\"_vType\":{\"fields\":[{\"name\":\"the entries! [877f12a8827e18f61222c6c8c5fb04a8]\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"GT\",\"typ\":{},\"index\":0},{\"name\":\"RNC\",\"typ\":{\"elementType\":{}},\"index\":1},{\"name\":\"DP\",\"typ\":{},\"index\":2},{\"name\":\"AD\",\"typ\":{\"elementType\":{}},\"index\":3},{\"name\":\"GQ\",\"typ\":{},\"index\":4},{\"name\":\"PL\",\"typ\":{\"elementType\":{}},\"index\":5}]}},\"index\":0}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_indexSpec\":{\"_relPath\":\"../../index\",\"_leafCodec\":{\"_eType\":{\"fields\":[{\"name\":\"first_idx\",\"typ\":{},\"index\":0},{\"name\":\"keys\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"fields\":[{\"name\":\"contig\",\"typ\":{},\"index\":0},{\"name\":\"position\",\"typ\":{},\"index\":1}],\"required\":true},\"index\":0}],\"required\":true},\"index\":0},{\"name\":\"offset\",\"typ\":{},\"index\":1},{\"name\":\"annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}],\"required\":true},\"index\":2}],\"required\":true},\"required\":true},\"index\":1}],\"required\":false},\"_vType\":{\"fields\":[{\"name\":\"first_idx\",\"typ\":{},\"index\":0},{\"name\":\"keys\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"index\":0},{\"name\":\"offset\",\"typ\":{},\"index\":1},{\"name\":\"annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]},\"index\":2}]}},\"index\":1}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_internalNodeCodec\":{\"_eType\":{\"fields\":[{\"name\":\"children\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"index_file_offset\",\"typ\":{},\"index\":0},{\"name\":\"first_idx\",\"typ\":{},\"index\":1},{\"name\":\"first_key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"fields\":[{\"name\":\"contig\",\"typ\":{},\"index\":0},{\"name\":\"position\",\"typ\":{},\"index\":1}],\"required\":true},\"index\":0}],\"required\":true},\"index\":2},{\"name\":\"first_record_offset\",\"typ\":{},\"index\":3},{\"name\":\"first_annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}],\"required\":true},\"index\":4}],\"required\":true},\"required\":true},\"index\":0}],\"required\":false},\"_vType\":{\"fields\":[{\"name\":\"children\",\"typ\":{\"elementType\":{\"fields\":[{\"name\":\"index_file_offset\",\"typ\":{},\"index\":0},{\"name\":\"first_idx\",\"typ\":{},\"index\":1},{\"name\":\"first_key\",\"typ\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"index\":2},{\"name\":\"first_record_offset\",\"typ\":{},\"index\":3},{\"name\":\"first_annotation\",\"typ\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]},\"index\":4}]}},\"index\":0}]},\"_bufferSpec\":{\"child\":{\"blockSize\":32768,\"child\":{\"blockSize\":32768,\"child\":{}}}}},\"_keyType\":{\"fields\":[{\"name\":\"locus\",\"typ\":{\"rgBc\":{}},\"index\":0}]},\"_annotationType\":{\"fields\":[{\"name\":\"entries_offset\",\"typ\":{},\"index\":0}]},\"_offsetField\":\"entries_offset\"},\"_partFiles\":[\"part-0-2-0-0-de4e832a-2b5f-07d0-d290-a2be64649dc3\"],\"_jRangeBounds\":[{\"start\":{},\"end\":{},\"includeStart\":true,\"includeEnd\":true}],\"_attrs\":{}}}}") (If (IsNA (Ref row)) (NA Struct{locus:Locus(GRCh38)}) (MakeStruct (locus (GetField locus (Ref row))))))))) (InsertFields (SelectFields () (Ref row)) None (locus (ApplySpecial showStr () String (GetField locus (Ref row)))))))) 2021-09-20 12:07:53 root: INFO: interpreting non compilable node: TableCollect 2021-09-20 12:07:53 root: INFO: decoder cache miss (0 hits, 1 misses, 0.000 2021-09-20 12:07:53 root: INFO: instruction count: 3: __C91etypeDecode. 2021-09-20 12:07:53 root: INFO: instruction count: 5: __C91etypeDecode.apply 2021-09-20 12:07:53 root: INFO: instruction count: 16: __C91etypeDecode.__m92DECODE_r_struct_of_END_TO_r_tuple_of_END 2021-09-20 12:07:53 root: INFO: decoder cache miss (0 hits, 2 misses, 0.000 2021-09-20 12:07:53 root: INFO: instruction count: 3: __C93etypeDecode. 2021-09-20 12:07:53 root: INFO: instruction count: 5: __C93etypeDecode.apply 2021-09-20 12:07:53 root: INFO: instruction count: 83: __C93etypeDecode.__m94DECODE_r_struct_of_r_struct_of_r_binaryANDr_int32ENDANDr_array_of_r_binaryANDo_binaryANDo_float64ANDo_array_of_r_binaryANDr_struct_of_o_array_of_o_float64ANDo_array_of_o_int32ANDo_array_of_o_int32ANDo_int32ENDEND_TO_r_tuple_of_r_locusEND 2021-09-20 12:07:53 root: INFO: instruction count: 22: __C93etypeDecode.__m95INPLACE_DECODE_r_struct_of_r_binaryANDr_int32END_TO_r_tuple_of_r_binaryANDr_int32END 2021-09-20 12:07:53 root: INFO: instruction count: 31: __C93etypeDecode.__m96INPLACE_DECODE_r_binary_TO_r_binary 2021-09-20 12:07:53 root: INFO: instruction count: 8: __C93etypeDecode.__m97INPLACE_DECODE_r_int32_TO_r_int32 2021-09-20 12:07:53 root: INFO: instruction count: 22: __C93etypeDecode.__m98SKIP_r_array_of_r_binary 2021-09-20 12:07:53 root: INFO: instruction count: 7: __C93etypeDecode.__m99SKIP_r_binary 2021-09-20 12:07:53 root: INFO: instruction count: 7: __C93etypeDecode.__m100SKIP_o_binary 2021-09-20 12:07:53 root: INFO: instruction count: 3: __C93etypeDecode.__m101SKIP_o_float64 2021-09-20 12:07:53 root: INFO: instruction count: 22: __C93etypeDecode.__m102SKIP_o_array_of_r_binary 2021-09-20 12:07:53 root: INFO: instruction count: 73: __C93etypeDecode.__m103SKIP_r_struct_of_o_array_of_o_float64ANDo_array_of_o_int32ANDo_array_of_o_int32ANDo_int32END 2021-09-20 12:07:53 root: INFO: instruction count: 51: __C93etypeDecode.__m104SKIP_o_array_of_o_float64 2021-09-20 12:07:53 root: INFO: instruction count: 51: __C93etypeDecode.__m105SKIP_o_array_of_o_int32 2021-09-20 12:07:53 root: INFO: instruction count: 3: __C93etypeDecode.__m106SKIP_o_int32 2021-09-20 12:07:53 MemoryStore: INFO: Block broadcast_3 stored as values in memory (estimated size 253.7 KB, free 11.5 GB) 2021-09-20 12:07:53 MemoryStore: INFO: Block broadcast_3_piece0 stored as bytes in memory (estimated size 24.3 KB, free 11.5 GB) 2021-09-20 12:07:53 BlockManagerInfo: INFO: Added broadcast_3_piece0 in memory on ip-10-60-47-217.eu-west-2.compute.internal:43001 (size: 24.3 KB, free: 11.5 GB) 2021-09-20 12:07:53 SparkContext: INFO: Created broadcast 3 from broadcast at SparkBackend.scala:250 2021-09-20 12:07:53 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 2: (InsertFields (In +PCStruct{locus:+PCLocus(GRCh38)} 0) None) 2021-09-20 12:07:53 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 1: (In +PCStruct{locus:+PCLocus(GRCh38)} 0) 2021-09-20 12:07:53 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 1: (In +PCStruct{locus:+PCLocus(GRCh38)} 0) 2021-09-20 12:07:53 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 1: (In +PCStruct{locus:+PCLocus(GRCh38)} 0) 2021-09-20 12:07:53 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 1: (In +PCStruct{locus:+PCLocus(GRCh38)} 0) 2021-09-20 12:07:53 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 1: (In +PCStruct{locus:+PCLocus(GRCh38)} 0) 2021-09-20 12:07:53 root: INFO: instruction count: 3: __C107Compiled. 2021-09-20 12:07:53 root: INFO: instruction count: 2: __C107Compiled.apply 2021-09-20 12:07:53 root: INFO: instruction count: 9: __C107Compiled.setPartitionIndex 2021-09-20 12:07:53 root: INFO: instruction count: 4: __C107Compiled.addPartitionRegion 2021-09-20 12:07:53 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 10: (Coalesce (If (IsNA (In +PCStruct{locus:+PCLocus(GRCh38)} 1)) (NA Struct{locus:Locus(GRCh38)}) (MakeStruct (locus (GetField locus (In +PCStruct{locus:+PCLocus(GRCh38)} 1))))) (Die Struct{locus:Locus(GRCh38)} -1 (Str "Internal e..."))) 2021-09-20 12:07:53 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 10: (Coalesce (If (IsNA (In +PCStruct{locus:+PCLocus(GRCh38)} 1)) (NA Struct{locus:Locus(GRCh38)}) (MakeStruct (locus (GetField locus (In +PCStruct{locus:+PCLocus(GRCh38)} 1))))) (Die Struct{locus:Locus(GRCh38)} -1 (Str "Internal e..."))) 2021-09-20 12:07:53 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 10: (Coalesce (If (IsNA (In +PCStruct{locus:+PCLocus(GRCh38)} 1)) (NA Struct{locus:Locus(GRCh38)}) (MakeStruct (locus (GetField locus (In +PCStruct{locus:+PCLocus(GRCh38)} 1))))) (Die Struct{locus:Locus(GRCh38)} -1 (Str "Internal e..."))) 2021-09-20 12:07:53 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 10: (Coalesce (If (IsNA (In +PCStruct{locus:+PCLocus(GRCh38)} 1)) (NA Struct{locus:Locus(GRCh38)}) (MakeStruct (locus (GetField locus (In +PCStruct{locus:+PCLocus(GRCh38)} 1))))) (Die Struct{locus:Locus(GRCh38)} -1 (Str "Internal e..."))) 2021-09-20 12:07:53 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 10: (Coalesce (If (IsNA (In +PCStruct{locus:+PCLocus(GRCh38)} 1)) (NA Struct{locus:Locus(GRCh38)}) (MakeStruct (locus (GetField locus (In +PCStruct{locus:+PCLocus(GRCh38)} 1))))) (Die Struct{locus:Locus(GRCh38)} -1 (Str "Internal e..."))) 2021-09-20 12:07:53 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 10: (Coalesce (If (IsNA (In +PCStruct{locus:+PCLocus(GRCh38)} 1)) (NA Struct{locus:Locus(GRCh38)}) (MakeStruct (locus (GetField locus (In +PCStruct{locus:+PCLocus(GRCh38)} 1))))) (Die Struct{locus:Locus(GRCh38)} -1 (Str "Internal e..."))) 2021-09-20 12:07:53 root: INFO: encoder cache hit 2021-09-20 12:07:53 MemoryStore: INFO: Block broadcast_4 stored as values in memory (estimated size 128.0 B, free 11.5 GB) 2021-09-20 12:07:53 MemoryStore: INFO: Block broadcast_4_piece0 stored as bytes in memory (estimated size 115.0 B, free 11.5 GB) 2021-09-20 12:07:53 BlockManagerInfo: INFO: Added broadcast_4_piece0 in memory on ip-10-60-47-217.eu-west-2.compute.internal:43001 (size: 115.0 B, free: 11.5 GB) 2021-09-20 12:07:53 SparkContext: INFO: Created broadcast 4 from broadcast at SparkBackend.scala:250 2021-09-20 12:07:53 root: INFO: instruction count: 3: __C110Compiled. 2021-09-20 12:07:53 root: INFO: instruction count: 98: __C110Compiled.apply 2021-09-20 12:07:53 root: INFO: instruction count: 10: __C110Compiled.__m113addIRIntermediate 2021-09-20 12:07:53 root: INFO: instruction count: 9: __C110Compiled.setPartitionIndex 2021-09-20 12:07:53 root: INFO: instruction count: 4: __C110Compiled.addPartitionRegion 2021-09-20 12:07:53 root: INFO: instruction count: 23: __C110Compiled.__m117DECODE_r_struct_of_r_binaryEND_TO_r_tuple_of_r_stringEND 2021-09-20 12:07:53 root: INFO: instruction count: 31: __C110Compiled.__m118INPLACE_DECODE_r_binary_TO_r_binary 2021-09-20 12:07:53 root: INFO: instruction count: 36: __C110Compiled.addLiterals 2021-09-20 12:07:53 SparkContext: INFO: Starting job: runJob at ContextRDD.scala:351 2021-09-20 12:07:53 DAGScheduler: INFO: [DNAX] Setting local property com.dnanexus.auth 2021-09-20 12:07:53 DAGScheduler: INFO: Unable to extract and set 'user' or 'containsPHI' in SparkSession 2021-09-20 12:07:53 DAGScheduler: INFO: Got job 0 (runJob at ContextRDD.scala:351) with 1 output partitions 2021-09-20 12:07:53 DAGScheduler: INFO: Final stage: ResultStage 0 (runJob at ContextRDD.scala:351) 2021-09-20 12:07:53 DAGScheduler: INFO: Parents of final stage: List() 2021-09-20 12:07:53 DAGScheduler: INFO: Missing parents: List() 2021-09-20 12:07:53 DAGScheduler: INFO: Submitting ResultStage 0 (MapPartitionsRDD[4] at mapPartitions at ContextRDD.scala:239), which has no missing parents 2021-09-20 12:07:53 MemoryStore: INFO: Block broadcast_5 stored as values in memory (estimated size 136.3 KB, free 11.5 GB) 2021-09-20 12:07:53 MemoryStore: INFO: Block broadcast_5_piece0 stored as bytes in memory (estimated size 46.3 KB, free 11.5 GB) 2021-09-20 12:07:53 BlockManagerInfo: INFO: Added broadcast_5_piece0 in memory on ip-10-60-47-217.eu-west-2.compute.internal:43001 (size: 46.3 KB, free: 11.5 GB) 2021-09-20 12:07:53 SparkContext: INFO: Created broadcast 5 from broadcast at DAGScheduler.scala:1268 2021-09-20 12:07:53 DAGScheduler: INFO: Submitting 1 missing tasks from ResultStage 0 (MapPartitionsRDD[4] at mapPartitions at ContextRDD.scala:239) (first 15 tasks are for partitions Vector(0)) 2021-09-20 12:07:53 TaskSchedulerImpl: INFO: Adding task set 0.0 with 1 tasks 2021-09-20 12:07:53 TaskSetManager: INFO: Starting task 0.0 in stage 0.0 (TID 0, ip-10-60-141-91.eu-west-2.compute.internal, executor 0, partition 0, PROCESS_LOCAL, 7900 bytes) 2021-09-20 12:07:53 BlockManagerInfo: INFO: Added broadcast_5_piece0 in memory on ip-10-60-141-91.eu-west-2.compute.internal:44000 (size: 46.3 KB, free: 11.5 GB) 2021-09-20 12:07:54 BlockManagerInfo: INFO: Added broadcast_3_piece0 in memory on ip-10-60-141-91.eu-west-2.compute.internal:44000 (size: 24.3 KB, free: 11.5 GB) 2021-09-20 12:07:54 BlockManagerInfo: INFO: Added broadcast_4_piece0 in memory on ip-10-60-141-91.eu-west-2.compute.internal:44000 (size: 115.0 B, free: 11.5 GB) 2021-09-20 12:07:54 TaskSetManager: INFO: Finished task 0.0 in stage 0.0 (TID 0) in 1059 ms on ip-10-60-141-91.eu-west-2.compute.internal (executor 0) (1/1) 2021-09-20 12:07:54 TaskSchedulerImpl: INFO: Removed TaskSet 0.0, whose tasks have all completed, from pool 2021-09-20 12:07:54 DAGScheduler: INFO: ResultStage 0 (runJob at ContextRDD.scala:351) finished in 1.115 s 2021-09-20 12:07:54 DAGScheduler: INFO: Job 0 finished: runJob at ContextRDD.scala:351, took 1.154276 s 2021-09-20 12:07:54 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 9: (Coalesce (InsertFields (SelectFields () (In +PCStruct{locus:+PCLocus(GRCh38)} 1)) None (locus (ApplySpecial showStr () String (GetField locus (In +PCStruct{locus:+PCLocus(GRCh38)} 1))))) (Die Struct{locus:String} -1 (Str "Internal e..."))) 2021-09-20 12:07:54 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 9: (Coalesce (InsertFields (SelectFields () (In +PCStruct{locus:+PCLocus(GRCh38)} 1)) None (locus (ApplySpecial showStr () String (GetField locus (In +PCStruct{locus:+PCLocus(GRCh38)} 1))))) (Die Struct{locus:String} -1 (Str "Internal e..."))) 2021-09-20 12:07:54 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 9: (Coalesce (InsertFields (SelectFields () (In +PCStruct{locus:+PCLocus(GRCh38)} 1)) None (locus (ApplySpecial showStr () String (GetField locus (In +PCStruct{locus:+PCLocus(GRCh38)} 1))))) (Die Struct{locus:String} -1 (Str "Internal e..."))) 2021-09-20 12:07:54 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 9: (Coalesce (InsertFields (SelectFields () (In +PCStruct{locus:+PCLocus(GRCh38)} 1)) None (locus (ApplySpecial showStr () String (GetField locus (In +PCStruct{locus:+PCLocus(GRCh38)} 1))))) (Die Struct{locus:String} -1 (Str "Internal e..."))) 2021-09-20 12:07:54 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 9: (Coalesce (InsertFields (SelectFields () (In +PCStruct{locus:+PCLocus(GRCh38)} 1)) None (locus (ApplySpecial showStr () String (GetField locus (In +PCStruct{locus:+PCLocus(GRCh38)} 1))))) (Die Struct{locus:String} -1 (Str "Internal e..."))) 2021-09-20 12:07:54 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 9: (Coalesce (InsertFields (SelectFields () (In +PCStruct{locus:+PCLocus(GRCh38)} 1)) None (locus (ApplySpecial showStr () String (GetField locus (In +PCStruct{locus:+PCLocus(GRCh38)} 1))))) (Die Struct{locus:String} -1 (Str "Internal e..."))) 2021-09-20 12:07:54 root: INFO: encoder cache hit 2021-09-20 12:07:54 MemoryStore: INFO: Block broadcast_6 stored as values in memory (estimated size 128.0 B, free 11.5 GB) 2021-09-20 12:07:54 MemoryStore: INFO: Block broadcast_6_piece0 stored as bytes in memory (estimated size 115.0 B, free 11.5 GB) 2021-09-20 12:07:54 BlockManagerInfo: INFO: Added broadcast_6_piece0 in memory on ip-10-60-47-217.eu-west-2.compute.internal:43001 (size: 115.0 B, free: 11.5 GB) 2021-09-20 12:07:54 SparkContext: INFO: Created broadcast 6 from broadcast at SparkBackend.scala:250 2021-09-20 12:07:54 root: INFO: instruction count: 3: __C119Compiled. 2021-09-20 12:07:54 root: INFO: instruction count: 158: __C119Compiled.apply 2021-09-20 12:07:54 root: INFO: instruction count: 23: __C119Compiled.__m129setup_null 2021-09-20 12:07:54 root: INFO: instruction count: 5: __C119Compiled.__m132addIRIntermediate 2021-09-20 12:07:54 root: INFO: instruction count: 9: __C119Compiled.setPartitionIndex 2021-09-20 12:07:54 root: INFO: instruction count: 4: __C119Compiled.addPartitionRegion 2021-09-20 12:07:54 root: INFO: instruction count: 23: __C119Compiled.__m136DECODE_r_struct_of_r_binaryEND_TO_r_tuple_of_r_stringEND 2021-09-20 12:07:54 root: INFO: instruction count: 31: __C119Compiled.__m137INPLACE_DECODE_r_binary_TO_r_binary 2021-09-20 12:07:54 root: INFO: instruction count: 36: __C119Compiled.addLiterals 2021-09-20 12:07:54 root: INFO: instruction count: 3: __C126RGContainer_GRCh38. 2021-09-20 12:07:54 root: INFO: instruction count: 15: __C126RGContainer_GRCh38. 2021-09-20 12:07:54 root: INFO: encoder cache miss (2 hits, 3 misses, 0.400) 2021-09-20 12:07:54 root: INFO: instruction count: 3: __C138etypeEncode. 2021-09-20 12:07:54 root: INFO: instruction count: 5: __C138etypeEncode.apply 2021-09-20 12:07:54 root: INFO: instruction count: 13: __C138etypeEncode.__m139ENCODE_r_tuple_of_r_stringEND_TO_r_struct_of_r_binaryEND 2021-09-20 12:07:54 root: INFO: instruction count: 16: __C138etypeEncode.__m140ENCODE_r_binary_TO_r_binary 2021-09-20 12:07:54 SparkContext: INFO: Starting job: collect at RVD.scala:762 2021-09-20 12:07:54 DAGScheduler: INFO: [DNAX] Setting local property com.dnanexus.auth 2021-09-20 12:07:54 DAGScheduler: INFO: Unable to extract and set 'user' or 'containsPHI' in SparkSession 2021-09-20 12:07:54 DAGScheduler: INFO: Got job 1 (collect at RVD.scala:762) with 1 output partitions 2021-09-20 12:07:54 DAGScheduler: INFO: Final stage: ResultStage 1 (collect at RVD.scala:762) 2021-09-20 12:07:54 DAGScheduler: INFO: Parents of final stage: List() 2021-09-20 12:07:54 DAGScheduler: INFO: Missing parents: List() 2021-09-20 12:07:54 DAGScheduler: INFO: Submitting ResultStage 1 (MapPartitionsRDD[10] at mapPartitions at ContextRDD.scala:160), which has no missing parents 2021-09-20 12:07:54 MemoryStore: INFO: Block broadcast_7 stored as values in memory (estimated size 305.8 KB, free 11.5 GB) 2021-09-20 12:07:54 MemoryStore: INFO: Block broadcast_7_piece0 stored as bytes in memory (estimated size 86.4 KB, free 11.5 GB) 2021-09-20 12:07:54 BlockManagerInfo: INFO: Added broadcast_7_piece0 in memory on ip-10-60-47-217.eu-west-2.compute.internal:43001 (size: 86.4 KB, free: 11.5 GB) 2021-09-20 12:07:54 SparkContext: INFO: Created broadcast 7 from broadcast at DAGScheduler.scala:1268 2021-09-20 12:07:54 DAGScheduler: INFO: Submitting 1 missing tasks from ResultStage 1 (MapPartitionsRDD[10] at mapPartitions at ContextRDD.scala:160) (first 15 tasks are for partitions Vector(0)) 2021-09-20 12:07:54 TaskSchedulerImpl: INFO: Adding task set 1.0 with 1 tasks 2021-09-20 12:07:54 TaskSetManager: INFO: Starting task 0.0 in stage 1.0 (TID 1, ip-10-60-141-91.eu-west-2.compute.internal, executor 0, partition 0, PROCESS_LOCAL, 7994 bytes) 2021-09-20 12:07:54 BlockManagerInfo: INFO: Added broadcast_7_piece0 in memory on ip-10-60-141-91.eu-west-2.compute.internal:44000 (size: 86.4 KB, free: 11.5 GB) 2021-09-20 12:07:54 BlockManagerInfo: INFO: Added broadcast_6_piece0 in memory on ip-10-60-141-91.eu-west-2.compute.internal:44000 (size: 115.0 B, free: 11.5 GB) 2021-09-20 12:07:54 TaskSetManager: INFO: Finished task 0.0 in stage 1.0 (TID 1) in 301 ms on ip-10-60-141-91.eu-west-2.compute.internal (executor 0) (1/1) 2021-09-20 12:07:54 TaskSchedulerImpl: INFO: Removed TaskSet 1.0, whose tasks have all completed, from pool 2021-09-20 12:07:54 DAGScheduler: INFO: ResultStage 1 (collect at RVD.scala:762) finished in 0.313 s 2021-09-20 12:07:54 DAGScheduler: INFO: Job 1 finished: collect at RVD.scala:762, took 0.317009 s 2021-09-20 12:07:54 root: INFO: decoder cache miss (0 hits, 3 misses, 0.000 2021-09-20 12:07:54 root: INFO: instruction count: 3: __C141etypeDecode. 2021-09-20 12:07:54 root: INFO: instruction count: 5: __C141etypeDecode.apply 2021-09-20 12:07:54 root: INFO: instruction count: 23: __C141etypeDecode.__m142DECODE_r_struct_of_r_binaryEND_TO_r_tuple_of_r_stringEND 2021-09-20 12:07:54 root: INFO: instruction count: 31: __C141etypeDecode.__m143INPLACE_DECODE_r_binary_TO_r_binary 2021-09-20 12:07:54 root: INFO: took 1.916s 2021-09-20 12:07:54 root: INFO: optimize optimize: relationalLowerer, after InterpretNonCompilable: before: IR size 2: (GetField rows (Literal Struct{rows:Array[Struct{locus:String}],global:Struct{}} )) 2021-09-20 12:07:54 root: INFO: optimize optimize: relationalLowerer, after InterpretNonCompilable: after: IR size 1: (Literal Array[Struct{locus:String}] ) 2021-09-20 12:07:54 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 2: (MakeTuple (0) (Literal Array[Struct{locus:String}] )) 2021-09-20 12:07:54 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 1: (Literal Tuple[Array[Struct{locus:String}]] ) 2021-09-20 12:07:54 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 1: (Literal Tuple[Array[Struct{locus:String}]] ) 2021-09-20 12:07:54 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 1: (Literal Tuple[Array[Struct{locus:String}]] ) 2021-09-20 12:07:54 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 1: (Literal Tuple[Array[Struct{locus:String}]] ) 2021-09-20 12:07:54 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 1: (Literal Tuple[Array[Struct{locus:String}]] ) 2021-09-20 12:07:54 root: INFO: encoder cache miss (2 hits, 4 misses, 0.333) 2021-09-20 12:07:54 root: INFO: instruction count: 3: __C153etypeEncode. 2021-09-20 12:07:54 root: INFO: instruction count: 5: __C153etypeEncode.apply 2021-09-20 12:07:54 root: INFO: instruction count: 9: __C153etypeEncode.__m154ENCODE_r_tuple_of_r_tuple_of_r_array_of_r_tuple_of_r_stringENDENDEND_TO_r_struct_of_r_struct_of_r_array_of_r_struct_of_r_binaryENDENDEND 2021-09-20 12:07:54 root: INFO: instruction count: 13: __C153etypeEncode.__m155ENCODE_r_tuple_of_r_array_of_r_tuple_of_r_binaryENDEND_TO_r_struct_of_r_array_of_r_struct_of_r_binaryENDEND 2021-09-20 12:07:54 root: INFO: instruction count: 49: __C153etypeEncode.__m156ENCODE_r_array_of_r_tuple_of_r_binaryEND_TO_r_array_of_r_struct_of_r_binaryEND 2021-09-20 12:07:54 root: INFO: instruction count: 13: __C153etypeEncode.__m157ENCODE_r_tuple_of_r_binaryEND_TO_r_struct_of_r_binaryEND 2021-09-20 12:07:54 root: INFO: instruction count: 16: __C153etypeEncode.__m158ENCODE_r_binary_TO_r_binary 2021-09-20 12:07:54 MemoryStore: INFO: Block broadcast_8 stored as values in memory (estimated size 280.0 B, free 11.5 GB) 2021-09-20 12:07:54 MemoryStore: INFO: Block broadcast_8_piece0 stored as bytes in memory (estimated size 129.0 B, free 11.5 GB) 2021-09-20 12:07:54 BlockManagerInfo: INFO: Added broadcast_8_piece0 in memory on ip-10-60-47-217.eu-west-2.compute.internal:43001 (size: 129.0 B, free: 11.5 GB) 2021-09-20 12:07:54 SparkContext: INFO: Created broadcast 8 from broadcast at SparkBackend.scala:250 2021-09-20 12:07:54 root: INFO: instruction count: 3: __C144Compiled. 2021-09-20 12:07:54 root: INFO: instruction count: 3: __C144Compiled.apply 2021-09-20 12:07:54 root: INFO: instruction count: 9: __C144Compiled.setPartitionIndex 2021-09-20 12:07:54 root: INFO: instruction count: 4: __C144Compiled.addPartitionRegion 2021-09-20 12:07:54 root: INFO: instruction count: 23: __C144Compiled.__m148DECODE_r_struct_of_r_struct_of_r_array_of_r_struct_of_r_binaryENDENDEND_TO_r_tuple_of_r_tuple_of_r_array_of_r_tuple_of_r_stringENDENDEND 2021-09-20 12:07:54 root: INFO: instruction count: 15: __C144Compiled.__m149INPLACE_DECODE_r_struct_of_r_array_of_r_struct_of_r_binaryENDEND_TO_r_tuple_of_r_array_of_r_tuple_of_r_binaryENDEND 2021-09-20 12:07:54 root: INFO: instruction count: 52: __C144Compiled.__m150INPLACE_DECODE_r_array_of_r_struct_of_r_binaryEND_TO_r_array_of_r_tuple_of_r_binaryEND 2021-09-20 12:07:54 root: INFO: instruction count: 15: __C144Compiled.__m151INPLACE_DECODE_r_struct_of_r_binaryEND_TO_r_tuple_of_r_binaryEND 2021-09-20 12:07:54 root: INFO: instruction count: 31: __C144Compiled.__m152INPLACE_DECODE_r_binary_TO_r_binary 2021-09-20 12:07:54 root: INFO: instruction count: 32: __C144Compiled.addLiterals 2021-09-20 12:07:54 root: INFO: finished execution of query hail_query_3 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON total 1.984s self 2.341ms children 1.981s %children 99.88% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR total 18.743ms self 0.016ms children 18.727ms %children 99.91% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/Verify total 0.015ms self 0.015ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation total 18.699ms self 1.972ms children 16.727ms %children 89.45% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize total 16.727ms self 0.063ms children 16.664ms %children 99.62% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.153ms self 0.153ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.617ms self 0.617ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 2.092ms self 2.092ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.951ms self 0.951ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.096ms self 0.096ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 6.324ms self 6.324ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.239ms self 0.239ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.210ms self 0.210ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.380ms self 0.380ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 1.043ms self 1.043ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.087ms self 0.087ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 1.694ms self 1.694ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.160ms self 0.160ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.165ms self 0.165ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.172ms self 0.172ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 1.029ms self 1.029ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.097ms self 0.097ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 1.156ms self 1.156ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, initial IR/Verify total 0.013ms self 0.013ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable total 2.017ms self 0.004ms children 2.012ms %children 99.79% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable/Verify total 0.006ms self 0.006ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable/LoweringTransformation total 1.949ms self 1.949ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/LowerMatrixToTable/Verify total 0.057ms self 0.057ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable total 25.181ms self 0.006ms children 25.175ms %children 99.98% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/Verify total 0.009ms self 0.009ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation total 25.153ms self 10.115ms children 15.038ms %children 59.79% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize total 15.038ms self 0.055ms children 14.983ms %children 99.63% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/FoldConstants total 0.126ms self 0.126ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.321ms self 0.321ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/Simplify total 1.604ms self 1.604ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardLets total 1.876ms self 1.876ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.104ms self 0.104ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/PruneDeadFields total 4.740ms self 4.740ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/FoldConstants total 0.193ms self 0.193ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.223ms self 0.223ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/Simplify total 0.239ms self 0.239ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardLets total 0.839ms self 0.839ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.088ms self 0.088ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/PruneDeadFields total 0.977ms self 0.977ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/FoldConstants total 0.155ms self 0.155ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.166ms self 0.166ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/Simplify total 0.140ms self 0.140ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardLets total 0.796ms self 0.796ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.089ms self 0.089ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/LoweringTransformation/Optimize/PruneDeadFields total 2.305ms self 2.305ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after LowerMatrixToTable/Verify total 0.013ms self 0.013ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable total 1.917s self 0.016ms children 1.917s %children 100.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/Verify total 0.021ms self 0.021ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation total 1.916s self 1.902s children 14.978ms %children 0.78% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR total 1.384ms self 0.009ms children 1.375ms %children 99.34% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.027ms self 0.027ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation total 1.344ms self 0.414ms children 0.930ms %children 69.19% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize total 0.930ms self 0.040ms children 0.890ms %children 95.74% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.137ms self 0.137ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.032ms self 0.032ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.060ms self 0.060ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.098ms self 0.098ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.018ms self 0.018ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.048ms self 0.048ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.364ms self 0.364ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.004ms self 0.004ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.014ms self 0.014ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.080ms self 0.080ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.011ms self 0.011ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.025ms self 0.025ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR total 0.019ms self 0.002ms children 0.017ms %children 90.41% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.006ms self 0.006ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/LoweringTransformation total 0.008ms self 0.008ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.003ms self 0.003ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR total 0.304ms self 0.002ms children 0.302ms %children 99.35% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.001ms self 0.001ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation total 0.299ms self 0.141ms children 0.159ms %children 52.95% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize total 0.159ms self 0.007ms children 0.152ms %children 95.82% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.023ms self 0.023ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.008ms self 0.008ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.017ms self 0.017ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 0.068ms self 0.068ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.010ms self 0.010ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 0.025ms self 0.025ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.002ms self 0.002ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs total 0.138ms self 0.002ms children 0.136ms %children 98.69% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/LoweringTransformation total 0.128ms self 0.128ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs total 0.294ms self 0.002ms children 0.291ms %children 99.22% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.001ms self 0.001ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation total 0.289ms self 0.147ms children 0.141ms %children 48.96% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize total 0.141ms self 0.006ms children 0.135ms %children 95.61% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.020ms self 0.020ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.006ms self 0.006ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.014ms self 0.014ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 0.062ms self 0.062ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.010ms self 0.010ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 0.024ms self 0.024ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.002ms self 0.002ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR total 2.723ms self 0.005ms children 2.718ms %children 99.82% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.018ms self 0.018ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation total 2.693ms self 0.596ms children 2.097ms %children 77.87% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize total 2.097ms self 0.045ms children 2.052ms %children 97.84% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.196ms self 0.196ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.035ms self 0.035ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.330ms self 0.330ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.438ms self 0.438ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.049ms self 0.049ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 1.004ms self 1.004ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.007ms self 0.007ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR total 0.067ms self 0.005ms children 0.062ms %children 92.44% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.016ms self 0.016ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/LoweringTransformation total 0.035ms self 0.035ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.011ms self 0.011ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR total 1.259ms self 0.003ms children 1.256ms %children 99.77% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation total 1.247ms self 0.471ms children 0.775ms %children 62.20% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize total 0.775ms self 0.016ms children 0.759ms %children 97.93% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.059ms self 0.059ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.033ms self 0.033ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.067ms self 0.067ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 0.371ms self 0.371ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.035ms self 0.035ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 0.194ms self 0.194ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.005ms self 0.005ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs total 1.286ms self 0.003ms children 1.283ms %children 99.74% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.012ms self 0.012ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/LoweringTransformation total 1.254ms self 1.254ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.016ms self 0.016ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs total 1.244ms self 0.003ms children 1.240ms %children 99.73% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.003ms self 0.003ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation total 1.232ms self 0.465ms children 0.767ms %children 62.29% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize total 0.767ms self 0.015ms children 0.752ms %children 98.03% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.055ms self 0.055ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.033ms self 0.033ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.066ms self 0.066ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 0.368ms self 0.368ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.037ms self 0.037ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 0.194ms self 0.194ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.005ms self 0.005ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR total 1.985ms self 0.013ms children 1.972ms %children 99.33% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.028ms self 0.028ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation total 1.937ms self 0.616ms children 1.321ms %children 68.18% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize total 1.321ms self 0.042ms children 1.279ms %children 96.81% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.221ms self 0.221ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.043ms self 0.043ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.089ms self 0.089ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.399ms self 0.399ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.040ms self 0.040ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.487ms self 0.487ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, initial IR/Verify total 0.007ms self 0.007ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR total 0.068ms self 0.003ms children 0.065ms %children 95.44% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.011ms self 0.011ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/LoweringTransformation total 0.026ms self 0.026ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/InlineApplyIR/Verify total 0.028ms self 0.028ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR total 1.918ms self 0.006ms children 1.912ms %children 99.70% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.003ms self 0.003ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation total 1.904ms self 0.523ms children 1.380ms %children 72.51% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize total 1.380ms self 0.022ms children 1.358ms %children 98.38% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.053ms self 0.053ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.037ms self 0.037ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.088ms self 0.088ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 0.786ms self 0.786ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.037ms self 0.037ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 0.356ms self 0.356ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after InlineApplyIR/Verify total 0.006ms self 0.006ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs total 0.546ms self 0.005ms children 0.541ms %children 99.14% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.011ms self 0.011ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/LoweringTransformation total 0.519ms self 0.519ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/LowerArrayAggsToRunAggs/Verify total 0.012ms self 0.012ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs total 1.745ms self 0.005ms children 1.739ms %children 99.70% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.003ms self 0.003ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation total 1.732ms self 0.457ms children 1.276ms %children 73.64% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize total 1.276ms self 0.021ms children 1.255ms %children 98.36% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.069ms self 0.069ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.041ms self 0.041ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.067ms self 0.067ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 0.370ms self 0.370ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.033ms self 0.033ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 0.675ms self 0.675ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/LoweringTransformation/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InterpretNonCompilable/Verify total 0.029ms self 0.029ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable total 0.764ms self 0.006ms children 0.759ms %children 99.28% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation total 0.751ms self 0.275ms children 0.477ms %children 63.46% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize total 0.477ms self 0.037ms children 0.439ms %children 92.16% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/FoldConstants total 0.163ms self 0.163ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.007ms self 0.007ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/Simplify total 0.024ms self 0.024ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardLets total 0.079ms self 0.079ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.012ms self 0.012ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/PruneDeadFields total 0.048ms self 0.048ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/FoldConstants total 0.022ms self 0.022ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.004ms self 0.004ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/Simplify total 0.014ms self 0.014ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardLets total 0.042ms self 0.042ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/ForwardRelationalLets total 0.007ms self 0.007ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/LoweringTransformation/Optimize/PruneDeadFields total 0.019ms self 0.019ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/optimize: relationalLowerer, after InterpretNonCompilable/Verify total 0.003ms self 0.003ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile total 17.032ms self 15.561ms children 1.471ms %children 8.63% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR total 0.492ms self 0.004ms children 0.488ms %children 99.12% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/Verify total 0.002ms self 0.002ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation total 0.483ms self 0.205ms children 0.278ms %children 57.57% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize total 0.278ms self 0.018ms children 0.260ms %children 93.44% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.092ms self 0.092ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.009ms self 0.009ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.010ms self 0.010ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.049ms self 0.049ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.007ms self 0.007ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.020ms self 0.020ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/FoldConstants total 0.016ms self 0.016ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.003ms self 0.003ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/Simplify total 0.006ms self 0.006ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardLets total 0.029ms self 0.029ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.005ms self 0.005ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/LoweringTransformation/Optimize/PruneDeadFields total 0.013ms self 0.013ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, initial IR/Verify total 0.002ms self 0.002ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR total 0.017ms self 0.002ms children 0.015ms %children 87.67% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR/Verify total 0.005ms self 0.005ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR/LoweringTransformation total 0.007ms self 0.007ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/InlineApplyIR/Verify total 0.003ms self 0.003ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR total 0.279ms self 0.003ms children 0.275ms %children 98.75% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/Verify total 0.001ms self 0.001ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation total 0.272ms self 0.152ms children 0.120ms %children 44.23% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize total 0.120ms self 0.007ms children 0.113ms %children 94.14% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/FoldConstants total 0.021ms self 0.021ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.007ms self 0.007ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/Simplify total 0.011ms self 0.011ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardLets total 0.048ms self 0.048ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/ForwardRelationalLets total 0.007ms self 0.007ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/LoweringTransformation/Optimize/PruneDeadFields total 0.019ms self 0.019ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after InlineApplyIR/Verify total 0.002ms self 0.002ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs total 0.402ms self 0.004ms children 0.397ms %children 98.94% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs/Verify total 0.004ms self 0.004ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs/LoweringTransformation total 0.387ms self 0.387ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/LowerArrayAggsToRunAggs/Verify total 0.006ms self 0.006ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs total 0.282ms self 0.003ms children 0.278ms %children 98.76% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.001ms self 0.001ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation total 0.276ms self 0.159ms children 0.117ms %children 42.38% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize total 0.117ms self 0.007ms children 0.110ms %children 94.14% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/FoldConstants total 0.021ms self 0.021ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ExtractIntervalFilters total 0.006ms self 0.006ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/Simplify total 0.010ms self 0.010ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardLets total 0.048ms self 0.048ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/ForwardRelationalLets total 0.007ms self 0.007ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/LoweringTransformation/Optimize/PruneDeadFields total 0.019ms self 0.019ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/Compile/optimize: compileLowerer, after LowerArrayAggsToRunAggs/Verify total 0.002ms self 0.002ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/InitializeCompiledFunction total 0.741ms self 0.741ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/RunCompiledFunction total 0.002ms self 0.002ms children 0.000ms %children 0.00% 2021-09-20 12:07:54 root: INFO: timing SparkBackend.executeJSON/convertRegionValueToAnnotation total 0.216ms self 0.216ms children 0.000ms %children 0.00%