2020-01-07 21:45:02 Hail: WARN: This Hail JAR was compiled for Spark 2.4.0, running with Spark 2.4.1. Compatibility is not guaranteed. 2020-01-07 21:45:02 SparkContext: INFO: Running Spark version 2.4.1 2020-01-07 21:45:02 SparkContext: INFO: Submitted application: Hail 2020-01-07 21:45:02 SparkContext: INFO: Spark configuration: spark.app.name=Hail spark.driver.extraClassPath=/fs/home/cliveterence.dar/.conda/envs/hail/lib/python3.7/site-packages/hail/hail-all-spark.jar spark.driver.maxResultSize=0 spark.executor.extraClassPath=./hail-all-spark.jar spark.hadoop.io.compression.codecs=org.apache.hadoop.io.compress.DefaultCodec,is.hail.io.compress.BGzipCodec,is.hail.io.compress.BGzipCodecTbi,org.apache.hadoop.io.compress.GzipCodec spark.hadoop.mapreduce.input.fileinputformat.split.minsize=0 spark.jars=file:///fs/home/cliveterence.dar/.conda/envs/hail/lib/python3.7/site-packages/hail/hail-all-spark.jar spark.kryo.registrator=is.hail.kryo.HailKryoRegistrator spark.kryoserializer.buffer.max=1g spark.logConf=true spark.master=local[*] spark.repl.local.jars=file:///fs/home/cliveterence.dar/.conda/envs/hail/lib/python3.7/site-packages/hail/hail-all-spark.jar spark.serializer=org.apache.spark.serializer.KryoSerializer spark.submit.deployMode=client spark.ui.showConsoleProgress=false 2020-01-07 21:45:02 SecurityManager: INFO: Changing view acls to: cliveterence.dar 2020-01-07 21:45:02 SecurityManager: INFO: Changing modify acls to: cliveterence.dar 2020-01-07 21:45:02 SecurityManager: INFO: Changing view acls groups to: 2020-01-07 21:45:02 SecurityManager: INFO: Changing modify acls groups to: 2020-01-07 21:45:02 SecurityManager: INFO: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(cliveterence.dar); groups with view permissions: Set(); users with modify permissions: Set(cliveterence.dar); groups with modify permissions: Set() 2020-01-07 21:45:02 Utils: INFO: Successfully started service 'sparkDriver' on port 45738. 2020-01-07 21:45:02 SparkEnv: INFO: Registering MapOutputTracker 2020-01-07 21:45:02 SparkEnv: INFO: Registering BlockManagerMaster 2020-01-07 21:45:02 BlockManagerMasterEndpoint: INFO: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information 2020-01-07 21:45:02 BlockManagerMasterEndpoint: INFO: BlockManagerMasterEndpoint up 2020-01-07 21:45:02 DiskBlockManager: INFO: Created local directory at /tmp/blockmgr-3af18786-5c1e-478f-8bed-76e62d5a625a 2020-01-07 21:45:02 MemoryStore: INFO: MemoryStore started with capacity 366.3 MB 2020-01-07 21:45:02 SparkEnv: INFO: Registering OutputCommitCoordinator 2020-01-07 21:45:02 log: INFO: Logging initialized @2410ms 2020-01-07 21:45:02 Server: INFO: jetty-9.3.z-SNAPSHOT, build timestamp: unknown, git hash: unknown 2020-01-07 21:45:02 Server: INFO: Started @2482ms 2020-01-07 21:45:02 AbstractConnector: INFO: Started ServerConnector@7c15b2a4{HTTP/1.1,[http/1.1]}{0.0.0.0:4040} 2020-01-07 21:45:02 Utils: INFO: Successfully started service 'SparkUI' on port 4040. 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@52dc699e{/jobs,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@63ab8388{/jobs/json,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@4554575e{/jobs/job,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@3ac59bf9{/jobs/job/json,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@54de8ea{/stages,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@62c4a1ba{/stages/json,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@261c0ccf{/stages/stage,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@1dfe5673{/stages/stage/json,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@5bc24b77{/stages/pool,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@696dacc8{/stages/pool/json,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@34ef2b56{/storage,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@5bcdbe76{/storage/json,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@129dcf8b{/storage/rdd,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@16122afc{/storage/rdd/json,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@5b6a16f3{/environment,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@6de22de6{/environment/json,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@11ff4646{/executors,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@16bdcb4f{/executors/json,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@1d6b9ae5{/executors/threadDump,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@611a09b3{/executors/threadDump/json,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@a3caea0{/static,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@6daa01f7{/,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@58b3dbd5{/api,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@7005f613{/jobs/job/kill,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@77f3bd61{/stages/stage/kill,null,AVAILABLE,@Spark} 2020-01-07 21:45:02 SparkUI: INFO: Bound SparkUI to 0.0.0.0, and started at http://sinin.kps.ku.ac.th:4040 2020-01-07 21:45:02 SparkContext: INFO: Added JAR file:///fs/home/cliveterence.dar/.conda/envs/hail/lib/python3.7/site-packages/hail/hail-all-spark.jar at spark://sinin.kps.ku.ac.th:45738/jars/hail-all-spark.jar with timestamp 1578408302774 2020-01-07 21:45:02 Executor: INFO: Starting executor ID driver on host localhost 2020-01-07 21:45:02 Utils: INFO: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 35729. 2020-01-07 21:45:02 NettyBlockTransferService: INFO: Server created on sinin.kps.ku.ac.th:35729 2020-01-07 21:45:02 BlockManager: INFO: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy 2020-01-07 21:45:02 BlockManagerMaster: INFO: Registering BlockManager BlockManagerId(driver, sinin.kps.ku.ac.th, 35729, None) 2020-01-07 21:45:02 BlockManagerMasterEndpoint: INFO: Registering block manager sinin.kps.ku.ac.th:35729 with 366.3 MB RAM, BlockManagerId(driver, sinin.kps.ku.ac.th, 35729, None) 2020-01-07 21:45:02 BlockManagerMaster: INFO: Registered BlockManager BlockManagerId(driver, sinin.kps.ku.ac.th, 35729, None) 2020-01-07 21:45:02 BlockManager: INFO: Initialized BlockManager: BlockManagerId(driver, sinin.kps.ku.ac.th, 35729, None) 2020-01-07 21:45:03 ContextHandler: INFO: Started o.s.j.s.ServletContextHandler@789f2257{/metrics/json,null,AVAILABLE,@Spark} 2020-01-07 21:45:03 Hail: INFO: Hail temporary directory: file:/tmp/hail.LHDIkkbbALYE 2020-01-07 21:45:03 Hail: INFO: SparkUI: http://sinin.kps.ku.ac.th:4040 2020-01-07 21:45:03 Hail: INFO: Running Hail version 0.2.30-2ae07d872f43 2020-01-07 21:45:03 SparkContext: WARN: Using an existing SparkContext; some configuration may not take effect. 2020-01-07 21:45:20 Hail: INFO: 1KG files found 2020-01-07 21:45:27 root: INFO: globbing path data/1kg.vcf.bgz returned 1 files: 1kg.vcf.bgz 2020-01-07 21:45:27 root: INFO: globbing path file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg.vcf.bgz returned 1 files: 1kg.vcf.bgz 2020-01-07 21:45:27 MemoryStore: INFO: Block broadcast_0 stored as values in memory (estimated size 237.3 KB, free 366.1 MB) 2020-01-07 21:45:27 MemoryStore: INFO: Block broadcast_0_piece0 stored as bytes in memory (estimated size 23.0 KB, free 366.0 MB) 2020-01-07 21:45:27 BlockManagerInfo: INFO: Added broadcast_0_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 23.0 KB, free: 366.3 MB) 2020-01-07 21:45:27 SparkContext: INFO: Created broadcast 0 from broadcast at HailContext.scala:618 2020-01-07 21:45:27 MemoryStore: INFO: Block broadcast_1 stored as values in memory (estimated size 24.2 KB, free 366.0 MB) 2020-01-07 21:45:27 MemoryStore: INFO: Block broadcast_1_piece0 stored as bytes in memory (estimated size 2.3 KB, free 366.0 MB) 2020-01-07 21:45:27 BlockManagerInfo: INFO: Added broadcast_1_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 2.3 KB, free: 366.3 MB) 2020-01-07 21:45:27 SparkContext: INFO: Created broadcast 1 from broadcast at SparkBackend.scala:21 2020-01-07 21:45:27 SparkContext: INFO: Starting job: foreach at LoadVCF.scala:1491 2020-01-07 21:45:27 DAGScheduler: INFO: Got job 0 (foreach at LoadVCF.scala:1491) with 1 output partitions 2020-01-07 21:45:27 DAGScheduler: INFO: Final stage: ResultStage 0 (foreach at LoadVCF.scala:1491) 2020-01-07 21:45:27 DAGScheduler: INFO: Parents of final stage: List() 2020-01-07 21:45:27 DAGScheduler: INFO: Missing parents: List() 2020-01-07 21:45:27 DAGScheduler: INFO: Submitting ResultStage 0 (ParallelCollectionRDD[0] at parallelize at LoadVCF.scala:1491), which has no missing parents 2020-01-07 21:45:27 MemoryStore: INFO: Block broadcast_2 stored as values in memory (estimated size 3.8 KB, free 366.0 MB) 2020-01-07 21:45:27 MemoryStore: INFO: Block broadcast_2_piece0 stored as bytes in memory (estimated size 2.3 KB, free 366.0 MB) 2020-01-07 21:45:27 BlockManagerInfo: INFO: Added broadcast_2_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 2.3 KB, free: 366.3 MB) 2020-01-07 21:45:27 SparkContext: INFO: Created broadcast 2 from broadcast at DAGScheduler.scala:1161 2020-01-07 21:45:27 DAGScheduler: INFO: Submitting 1 missing tasks from ResultStage 0 (ParallelCollectionRDD[0] at parallelize at LoadVCF.scala:1491) (first 15 tasks are for partitions Vector(0)) 2020-01-07 21:45:27 TaskSchedulerImpl: INFO: Adding task set 0.0 with 1 tasks 2020-01-07 21:45:28 TaskSetManager: INFO: Starting task 0.0 in stage 0.0 (TID 0, localhost, executor driver, partition 0, PROCESS_LOCAL, 7721 bytes) 2020-01-07 21:45:28 Executor: INFO: Running task 0.0 in stage 0.0 (TID 0) 2020-01-07 21:45:28 Executor: INFO: Finished task 0.0 in stage 0.0 (TID 0). 666 bytes result sent to driver 2020-01-07 21:45:28 TaskSetManager: INFO: Finished task 0.0 in stage 0.0 (TID 0) in 112 ms on localhost (executor driver) (1/1) 2020-01-07 21:45:28 TaskSchedulerImpl: INFO: Removed TaskSet 0.0, whose tasks have all completed, from pool 2020-01-07 21:45:28 DAGScheduler: INFO: ResultStage 0 (foreach at LoadVCF.scala:1491) finished in 0.296 s 2020-01-07 21:45:28 DAGScheduler: INFO: Job 0 finished: foreach at LoadVCF.scala:1491, took 0.340927 s 2020-01-07 21:45:28 MemoryStore: INFO: Block broadcast_3 stored as values in memory (estimated size 33.9 KB, free 366.0 MB) 2020-01-07 21:45:28 MemoryStore: INFO: Block broadcast_3_piece0 stored as bytes in memory (estimated size 2.9 KB, free 366.0 MB) 2020-01-07 21:45:28 BlockManagerInfo: INFO: Added broadcast_3_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 2.9 KB, free: 366.3 MB) 2020-01-07 21:45:28 SparkContext: INFO: Created broadcast 3 from broadcast at SparkBackend.scala:21 2020-01-07 21:45:28 root: INFO: RegionPool: initialized for thread 28: Thread-5 2020-01-07 21:45:28 root: INFO: optimize Lowerer, initial IR: before: IR size 2: (MatrixWrite "{\"name\":\"MatrixNativeWriter\",\"path\":\"data/1kg.mt\",\"overwrite\":true,\"stageLocally\":false,\"codecSpecJSONStr\":null,\"partitions\":null,\"partitionsTypeStr\":null}" (MatrixRead None False False "{\"name\":\"MatrixVCFReader\",\"files\":[\"data/1kg.vcf.bgz\"],\"callFields\":[\"PGT\"],\"entryFloatTypeName\":\"Float64\",\"rg\":\"GRCh37\",\"contigRecoding\":{},\"arrayElementsRequired\":true,\"skipInvalidLoci\":false,\"gzAsBGZ\":false,\"forceGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"partitionsJSON\":null}")) 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 2.000ms, total 16.846ms 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 3.630ms, total 21.277ms 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 7.492ms, total 29.102ms 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 6.792ms, total 36.242ms 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 1.145ms, total 37.722ms 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 13.480ms, total 51.599ms 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 37.319ms, total 51.814ms, tagged coverage 92.6 2020-01-07 21:45:28 root: INFO: optimize Lowerer, initial IR: after: IR size 2: (MatrixWrite "{\"name\":\"MatrixNativeWriter\",\"path\":\"data/1kg.mt\",\"overwrite\":true,\"stageLocally\":false,\"codecSpecJSONStr\":null,\"partitions\":null,\"partitionsTypeStr\":null}" (MatrixRead None False False "{\"name\":\"MatrixVCFReader\",\"files\":[\"data/1kg.vcf.bgz\"],\"callFields\":[\"PGT\"],\"entryFloatTypeName\":\"Float64\",\"rg\":\"GRCh37\",\"contigRecoding\":{},\"arrayElementsRequired\":true,\"skipInvalidLoci\":false,\"gzAsBGZ\":false,\"forceGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"partitionsJSON\":null}")) 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- Verify : 0.342ms, total 54.554ms 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- LoweringTransformation : 2.371ms, total 57.243ms 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- Verify : 0.338ms, total 57.912ms 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable : 4.127ms, total 58.135ms, tagged coverage 73.9 2020-01-07 21:45:28 root: INFO: optimize LowerMatrixToTable, post Lowering: before: IR size 2: (TableWrite "{\"writer\":{\"path\":\"data/1kg.mt\",\"overwrite\":true,\"stageLocally\":false,\"codecSpecJSONStr\":null,\"partitions\":null,\"partitionsTypeStr\":null},\"colsFieldName\":\"__cols\",\"entriesFieldName\":\"the entries! [877f12a8827e18f61222c6c8c5fb04a8]\",\"colKey\":[\"s\"]}" (TableRead None False "{\"files\":[\"data/1kg.vcf.bgz\"],\"callFields\":[\"PGT\"],\"entryFloatTypeName\":\"Float64\",\"rg\":\"GRCh37\",\"contigRecoding\":{},\"arrayElementsRequired\":true,\"skipInvalidLoci\":false,\"gzAsBGZ\":false,\"forceGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"partitionsJSON\":null}")) 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.073ms, total 64.597ms 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.139ms, total 64.978ms 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 3.033ms, total 68.167ms 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 1.181ms, total 69.556ms 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.034ms, total 69.799ms 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 1.044ms, total 71.025ms 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 6.751ms, total 71.269ms, tagged coverage 81.5 2020-01-07 21:45:28 root: INFO: optimize LowerMatrixToTable, post Lowering: after: IR size 2: (TableWrite "{\"writer\":{\"path\":\"data/1kg.mt\",\"overwrite\":true,\"stageLocally\":false,\"codecSpecJSONStr\":null,\"partitions\":null,\"partitionsTypeStr\":null},\"colsFieldName\":\"__cols\",\"entriesFieldName\":\"the entries! [877f12a8827e18f61222c6c8c5fb04a8]\",\"colKey\":[\"s\"]}" (TableRead None False "{\"files\":[\"data/1kg.vcf.bgz\"],\"callFields\":[\"PGT\"],\"entryFloatTypeName\":\"Float64\",\"rg\":\"GRCh37\",\"contigRecoding\":{},\"arrayElementsRequired\":true,\"skipInvalidLoci\":false,\"gzAsBGZ\":false,\"forceGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"partitionsJSON\":null}")) 2020-01-07 21:45:28 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- Verify : 0.009ms, total 72.354ms 2020-01-07 21:45:28 root: INFO: interpreting non compilable node: TableWrite 2020-01-07 21:45:28 MemoryStore: INFO: Block broadcast_4 stored as values in memory (estimated size 237.4 KB, free 365.7 MB) 2020-01-07 21:45:28 MemoryStore: INFO: Block broadcast_4_piece0 stored as bytes in memory (estimated size 23.0 KB, free 365.7 MB) 2020-01-07 21:45:28 BlockManagerInfo: INFO: Added broadcast_4_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 23.0 KB, free: 366.2 MB) 2020-01-07 21:45:28 SparkContext: INFO: Created broadcast 4 from textFile at RichSparkContext.scala:16 2020-01-07 21:45:28 FileInputFormat: INFO: Total input paths to process : 1 2020-01-07 21:45:28 SparkContext: INFO: Starting job: collect at ContextRDD.scala:223 2020-01-07 21:45:28 DAGScheduler: INFO: Got job 1 (collect at ContextRDD.scala:223) with 2 output partitions 2020-01-07 21:45:28 DAGScheduler: INFO: Final stage: ResultStage 1 (collect at ContextRDD.scala:223) 2020-01-07 21:45:28 DAGScheduler: INFO: Parents of final stage: List() 2020-01-07 21:45:28 DAGScheduler: INFO: Missing parents: List() 2020-01-07 21:45:28 DAGScheduler: INFO: Submitting ResultStage 1 (MapPartitionsRDD[8] at mapPartitions at ContextRDD.scala:217), which has no missing parents 2020-01-07 21:45:28 MemoryStore: INFO: Block broadcast_5 stored as values in memory (estimated size 17.5 KB, free 365.7 MB) 2020-01-07 21:45:28 MemoryStore: INFO: Block broadcast_5_piece0 stored as bytes in memory (estimated size 8.7 KB, free 365.7 MB) 2020-01-07 21:45:28 BlockManagerInfo: INFO: Added broadcast_5_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 8.7 KB, free: 366.2 MB) 2020-01-07 21:45:28 SparkContext: INFO: Created broadcast 5 from broadcast at DAGScheduler.scala:1161 2020-01-07 21:45:28 DAGScheduler: INFO: Submitting 2 missing tasks from ResultStage 1 (MapPartitionsRDD[8] at mapPartitions at ContextRDD.scala:217) (first 15 tasks are for partitions Vector(0, 1)) 2020-01-07 21:45:28 TaskSchedulerImpl: INFO: Adding task set 1.0 with 2 tasks 2020-01-07 21:45:28 TaskSetManager: INFO: Starting task 0.0 in stage 1.0 (TID 1, localhost, executor driver, partition 0, PROCESS_LOCAL, 7927 bytes) 2020-01-07 21:45:28 TaskSetManager: INFO: Starting task 1.0 in stage 1.0 (TID 2, localhost, executor driver, partition 1, PROCESS_LOCAL, 7927 bytes) 2020-01-07 21:45:28 Executor: INFO: Running task 0.0 in stage 1.0 (TID 1) 2020-01-07 21:45:28 Executor: INFO: Running task 1.0 in stage 1.0 (TID 2) 2020-01-07 21:45:28 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg.vcf.bgz:8429631+8429631 2020-01-07 21:45:28 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg.vcf.bgz:0+8429631 2020-01-07 21:45:28 CodecPool: INFO: Got brand-new decompressor [.bgz] 2020-01-07 21:45:28 CodecPool: INFO: Got brand-new decompressor [.bgz] 2020-01-07 21:45:28 root: INFO: RegionPool: initialized for thread 99: Executor task launch worker for task 2 2020-01-07 21:45:28 root: INFO: RegionPool: initialized for thread 97: Executor task launch worker for task 1 2020-01-07 21:45:28 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), thread 99: Executor task launch worker for task 2 2020-01-07 21:45:28 root: INFO: RegionPool: REPORT_THRESHOLD: 512.0K allocated (512.0K blocks / 0 chunks), thread 99: Executor task launch worker for task 2 2020-01-07 21:45:28 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), thread 97: Executor task launch worker for task 1 2020-01-07 21:45:28 root: INFO: RegionPool: REPORT_THRESHOLD: 512.0K allocated (512.0K blocks / 0 chunks), thread 97: Executor task launch worker for task 1 2020-01-07 21:45:28 root: INFO: RegionPool: REPORT_THRESHOLD: 1.0M allocated (1.0M blocks / 0 chunks), thread 99: Executor task launch worker for task 2 2020-01-07 21:45:28 root: INFO: RegionPool: REPORT_THRESHOLD: 1.0M allocated (1.0M blocks / 0 chunks), thread 97: Executor task launch worker for task 1 2020-01-07 21:45:28 Executor: INFO: Finished task 1.0 in stage 1.0 (TID 2). 1603 bytes result sent to driver 2020-01-07 21:45:28 TaskSetManager: INFO: Finished task 1.0 in stage 1.0 (TID 2) in 487 ms on localhost (executor driver) (1/2) 2020-01-07 21:45:28 Executor: INFO: Finished task 0.0 in stage 1.0 (TID 1). 1604 bytes result sent to driver 2020-01-07 21:45:29 TaskSetManager: INFO: Finished task 0.0 in stage 1.0 (TID 1) in 511 ms on localhost (executor driver) (2/2) 2020-01-07 21:45:29 TaskSchedulerImpl: INFO: Removed TaskSet 1.0, whose tasks have all completed, from pool 2020-01-07 21:45:29 DAGScheduler: INFO: ResultStage 1 (collect at ContextRDD.scala:223) finished in 0.523 s 2020-01-07 21:45:29 DAGScheduler: INFO: Job 1 finished: collect at ContextRDD.scala:223, took 0.529791 s 2020-01-07 21:45:29 Hail: INFO: Coerced sorted dataset 2020-01-07 21:45:29 root: INFO: encoder cache miss (0 hits, 1 misses, 0.000) 2020-01-07 21:45:29 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_0. 2020-01-07 21:45:29 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_0.apply 2020-01-07 21:45:29 root: INFO: instruction count: 15: is/hail/codegen/generated/C_etypeEncode_0.m0_ENCODE_o_tuple_of_o_locusANDo_array_of_o_stringANDo_stringANDo_float64ANDo_set_of_o_stringANDo_tuple_of_o_array_of_o_int32ANDo_array_of_o_float64ANDo_int32ANDo_float64ANDo_float64ANDo_int32ANDo_boolANDo_float64ANDo_float64ANDo_float64ANDo_array_of_o_int32ANDo_array_of_o_float64ANDo_float64ANDo_int32ANDo_float64ANDo_float64ANDo_float64ANDo_stringENDANDo_array_of_o_tuple_of_o_callANDo_array_of_r_int32ANDo_int32ANDo_int32ANDo_array_of_r_int32ENDEND_TO_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryANDo_binaryANDo_float64ANDo_array_of_o_binaryANDo_struct_of_o_array_of_o_int32ANDo_array_of_o_float64ANDo_int32ANDo_float64ANDo_float64ANDo_int32ANDo_boolANDo_float64ANDo_float64ANDo_float64ANDo_array_of_o_int32ANDo_array_of_o_float64ANDo_float64ANDo_int32ANDo_float64ANDo_float64ANDo_float64ANDo_binaryENDEND 2020-01-07 21:45:29 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_0.m1_missingbits_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 49: is/hail/codegen/generated/C_etypeEncode_0.m2_missingbits_group_1 2020-01-07 21:45:29 root: INFO: instruction count: 107: is/hail/codegen/generated/C_etypeEncode_0.m3_write_fields_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_0.m4_ENCODE_o_tuple_of_r_binaryANDr_int32END_TO_o_struct_of_r_binaryANDr_int32END 2020-01-07 21:45:29 root: INFO: instruction count: 33: is/hail/codegen/generated/C_etypeEncode_0.m5_write_fields_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 16: is/hail/codegen/generated/C_etypeEncode_0.m6_ENCODE_r_binary_TO_r_binary 2020-01-07 21:45:29 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_0.m7_ENCODE_r_int32_TO_r_int32 2020-01-07 21:45:29 root: INFO: instruction count: 76: is/hail/codegen/generated/C_etypeEncode_0.m8_ENCODE_o_array_of_o_binary_TO_o_array_of_o_binary 2020-01-07 21:45:29 root: INFO: instruction count: 16: is/hail/codegen/generated/C_etypeEncode_0.m9_ENCODE_o_binary_TO_o_binary 2020-01-07 21:45:29 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_0.m10_ENCODE_o_float64_TO_o_float64 2020-01-07 21:45:29 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_0.m11_ENCODE_o_tuple_of_o_array_of_o_int32ANDo_array_of_o_float64ANDo_int32ANDo_float64ANDo_float64ANDo_int32ANDo_boolANDo_float64ANDo_float64ANDo_float64ANDo_array_of_o_int32ANDo_array_of_o_float64ANDo_float64ANDo_int32ANDo_float64ANDo_float64ANDo_float64ANDo_binaryEND_TO_o_struct_of_o_array_of_o_int32ANDo_array_of_o_float64ANDo_int32ANDo_float64ANDo_float64ANDo_int32ANDo_boolANDo_float64ANDo_float64ANDo_float64ANDo_array_of_o_int32ANDo_array_of_o_float64ANDo_float64ANDo_int32ANDo_float64ANDo_float64ANDo_float64ANDo_binaryEND 2020-01-07 21:45:29 root: INFO: instruction count: 327: is/hail/codegen/generated/C_etypeEncode_0.m12_write_fields_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 76: is/hail/codegen/generated/C_etypeEncode_0.m13_ENCODE_o_array_of_o_int32_TO_o_array_of_o_int32 2020-01-07 21:45:29 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_0.m14_ENCODE_o_int32_TO_o_int32 2020-01-07 21:45:29 root: INFO: instruction count: 76: is/hail/codegen/generated/C_etypeEncode_0.m15_ENCODE_o_array_of_o_float64_TO_o_array_of_o_float64 2020-01-07 21:45:29 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_0.m16_ENCODE_o_bool_TO_o_bool 2020-01-07 21:45:29 root: INFO: encoder cache miss (0 hits, 2 misses, 0.000) 2020-01-07 21:45:29 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_1. 2020-01-07 21:45:29 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_1.apply 2020-01-07 21:45:29 root: INFO: instruction count: 15: is/hail/codegen/generated/C_etypeEncode_1.m0_ENCODE_o_tuple_of_o_locusANDo_array_of_o_stringANDo_stringANDo_float64ANDo_set_of_o_stringANDo_tuple_of_o_array_of_o_int32ANDo_array_of_o_float64ANDo_int32ANDo_float64ANDo_float64ANDo_int32ANDo_boolANDo_float64ANDo_float64ANDo_float64ANDo_array_of_o_int32ANDo_array_of_o_float64ANDo_float64ANDo_int32ANDo_float64ANDo_float64ANDo_float64ANDo_stringENDANDo_array_of_o_tuple_of_o_callANDo_array_of_r_int32ANDo_int32ANDo_int32ANDo_array_of_r_int32ENDEND_TO_o_struct_of_o_array_of_o_struct_of_o_int32ANDo_array_of_r_int32ANDo_int32ANDo_int32ANDo_array_of_r_int32ENDEND 2020-01-07 21:45:29 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_1.m1_missingbits_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 14: is/hail/codegen/generated/C_etypeEncode_1.m2_missingbits_group_1 2020-01-07 21:45:29 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_1.m3_write_fields_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 74: is/hail/codegen/generated/C_etypeEncode_1.m4_ENCODE_o_array_of_o_tuple_of_o_int32ANDo_array_of_r_int32ANDo_int32ANDo_int32ANDo_array_of_r_int32END_TO_o_array_of_o_struct_of_o_int32ANDo_array_of_r_int32ANDo_int32ANDo_int32ANDo_array_of_r_int32END 2020-01-07 21:45:29 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_1.m5_ENCODE_o_tuple_of_o_int32ANDo_array_of_r_int32ANDo_int32ANDo_int32ANDo_array_of_r_int32END_TO_o_struct_of_o_int32ANDo_array_of_r_int32ANDo_int32ANDo_int32ANDo_array_of_r_int32END 2020-01-07 21:45:29 root: INFO: instruction count: 93: is/hail/codegen/generated/C_etypeEncode_1.m6_write_fields_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_1.m7_ENCODE_o_int32_TO_o_int32 2020-01-07 21:45:29 root: INFO: instruction count: 49: is/hail/codegen/generated/C_etypeEncode_1.m8_ENCODE_o_array_of_r_int32_TO_o_array_of_r_int32 2020-01-07 21:45:29 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_1.m9_ENCODE_r_int32_TO_r_int32 2020-01-07 21:45:29 root: INFO: encoder cache miss (0 hits, 3 misses, 0.000) 2020-01-07 21:45:29 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_2. 2020-01-07 21:45:29 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_2.apply 2020-01-07 21:45:29 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_2.m0_ENCODE_o_tuple_of_r_int64ANDr_array_of_r_tuple_of_o_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_o_int64ENDENDEND_TO_o_struct_of_r_int64ANDr_array_of_r_struct_of_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDENDEND 2020-01-07 21:45:29 root: INFO: instruction count: 33: is/hail/codegen/generated/C_etypeEncode_2.m1_write_fields_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_2.m2_ENCODE_r_int64_TO_r_int64 2020-01-07 21:45:29 root: INFO: instruction count: 47: is/hail/codegen/generated/C_etypeEncode_2.m3_ENCODE_r_array_of_r_tuple_of_o_tuple_of_o_tuple_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_tuple_of_o_int64ENDEND_TO_r_array_of_r_struct_of_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDEND 2020-01-07 21:45:29 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_2.m4_ENCODE_r_tuple_of_o_tuple_of_o_tuple_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_tuple_of_o_int64ENDEND_TO_r_struct_of_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDEND 2020-01-07 21:45:29 root: INFO: instruction count: 47: is/hail/codegen/generated/C_etypeEncode_2.m5_write_fields_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_2.m6_ENCODE_o_tuple_of_o_tuple_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryEND_TO_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryEND 2020-01-07 21:45:29 root: INFO: instruction count: 37: is/hail/codegen/generated/C_etypeEncode_2.m7_write_fields_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_2.m8_ENCODE_o_tuple_of_r_binaryANDr_int32END_TO_o_struct_of_r_binaryANDr_int32END 2020-01-07 21:45:29 root: INFO: instruction count: 33: is/hail/codegen/generated/C_etypeEncode_2.m9_write_fields_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 16: is/hail/codegen/generated/C_etypeEncode_2.m10_ENCODE_r_binary_TO_r_binary 2020-01-07 21:45:29 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_2.m11_ENCODE_r_int32_TO_r_int32 2020-01-07 21:45:29 root: INFO: instruction count: 76: is/hail/codegen/generated/C_etypeEncode_2.m12_ENCODE_o_array_of_o_binary_TO_o_array_of_o_binary 2020-01-07 21:45:29 root: INFO: instruction count: 16: is/hail/codegen/generated/C_etypeEncode_2.m13_ENCODE_o_binary_TO_o_binary 2020-01-07 21:45:29 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_2.m14_ENCODE_r_tuple_of_o_int64END_TO_r_struct_of_o_int64END 2020-01-07 21:45:29 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_2.m15_write_fields_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_2.m16_ENCODE_o_int64_TO_o_int64 2020-01-07 21:45:29 root: INFO: encoder cache miss (0 hits, 4 misses, 0.000) 2020-01-07 21:45:29 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_3. 2020-01-07 21:45:29 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_3.apply 2020-01-07 21:45:29 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_3.m0_ENCODE_o_tuple_of_r_array_of_r_tuple_of_r_int64ANDr_int64ANDo_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_o_int64ENDENDEND_TO_o_struct_of_r_array_of_r_struct_of_r_int64ANDr_int64ANDo_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDENDEND 2020-01-07 21:45:29 root: INFO: instruction count: 18: is/hail/codegen/generated/C_etypeEncode_3.m1_write_fields_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 47: is/hail/codegen/generated/C_etypeEncode_3.m2_ENCODE_r_array_of_r_tuple_of_r_int64ANDr_int64ANDo_tuple_of_o_tuple_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_tuple_of_o_int64ENDEND_TO_r_array_of_r_struct_of_r_int64ANDr_int64ANDo_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDEND 2020-01-07 21:45:29 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_3.m3_ENCODE_r_tuple_of_r_int64ANDr_int64ANDo_tuple_of_o_tuple_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_tuple_of_o_int64ENDEND_TO_r_struct_of_r_int64ANDr_int64ANDo_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDEND 2020-01-07 21:45:29 root: INFO: instruction count: 77: is/hail/codegen/generated/C_etypeEncode_3.m4_write_fields_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_3.m5_ENCODE_r_int64_TO_r_int64 2020-01-07 21:45:29 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_3.m6_ENCODE_o_tuple_of_o_tuple_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryEND_TO_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryEND 2020-01-07 21:45:29 root: INFO: instruction count: 37: is/hail/codegen/generated/C_etypeEncode_3.m7_write_fields_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_3.m8_ENCODE_o_tuple_of_r_binaryANDr_int32END_TO_o_struct_of_r_binaryANDr_int32END 2020-01-07 21:45:29 root: INFO: instruction count: 33: is/hail/codegen/generated/C_etypeEncode_3.m9_write_fields_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 16: is/hail/codegen/generated/C_etypeEncode_3.m10_ENCODE_r_binary_TO_r_binary 2020-01-07 21:45:29 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_3.m11_ENCODE_r_int32_TO_r_int32 2020-01-07 21:45:29 root: INFO: instruction count: 76: is/hail/codegen/generated/C_etypeEncode_3.m12_ENCODE_o_array_of_o_binary_TO_o_array_of_o_binary 2020-01-07 21:45:29 root: INFO: instruction count: 16: is/hail/codegen/generated/C_etypeEncode_3.m13_ENCODE_o_binary_TO_o_binary 2020-01-07 21:45:29 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_3.m14_ENCODE_r_tuple_of_o_int64END_TO_r_struct_of_o_int64END 2020-01-07 21:45:29 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_3.m15_write_fields_group_0 2020-01-07 21:45:29 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_3.m16_ENCODE_o_int64_TO_o_int64 2020-01-07 21:45:29 SparkContext: INFO: Starting job: collect at ContextRDD.scala:223 2020-01-07 21:45:29 DAGScheduler: INFO: Got job 2 (collect at ContextRDD.scala:223) with 2 output partitions 2020-01-07 21:45:29 DAGScheduler: INFO: Final stage: ResultStage 2 (collect at ContextRDD.scala:223) 2020-01-07 21:45:29 DAGScheduler: INFO: Parents of final stage: List() 2020-01-07 21:45:29 DAGScheduler: INFO: Missing parents: List() 2020-01-07 21:45:29 DAGScheduler: INFO: Submitting ResultStage 2 (MapPartitionsRDD[11] at mapPartitions at ContextRDD.scala:217), which has no missing parents 2020-01-07 21:45:29 MemoryStore: INFO: Block broadcast_6 stored as values in memory (estimated size 58.2 KB, free 365.6 MB) 2020-01-07 21:45:29 MemoryStore: INFO: Block broadcast_6_piece0 stored as bytes in memory (estimated size 25.2 KB, free 365.6 MB) 2020-01-07 21:45:29 BlockManagerInfo: INFO: Added broadcast_6_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 25.2 KB, free: 366.2 MB) 2020-01-07 21:45:29 SparkContext: INFO: Created broadcast 6 from broadcast at DAGScheduler.scala:1161 2020-01-07 21:45:29 DAGScheduler: INFO: Submitting 2 missing tasks from ResultStage 2 (MapPartitionsRDD[11] at mapPartitions at ContextRDD.scala:217) (first 15 tasks are for partitions Vector(0, 1)) 2020-01-07 21:45:29 TaskSchedulerImpl: INFO: Adding task set 2.0 with 2 tasks 2020-01-07 21:45:29 TaskSetManager: INFO: Starting task 0.0 in stage 2.0 (TID 3, localhost, executor driver, partition 0, PROCESS_LOCAL, 7927 bytes) 2020-01-07 21:45:29 TaskSetManager: INFO: Starting task 1.0 in stage 2.0 (TID 4, localhost, executor driver, partition 1, PROCESS_LOCAL, 7927 bytes) 2020-01-07 21:45:29 Executor: INFO: Running task 1.0 in stage 2.0 (TID 4) 2020-01-07 21:45:29 Executor: INFO: Running task 0.0 in stage 2.0 (TID 3) 2020-01-07 21:45:29 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg.vcf.bgz:8429631+8429631 2020-01-07 21:45:29 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg.vcf.bgz:0+8429631 2020-01-07 21:45:29 CodecPool: INFO: Got brand-new decompressor [.bgz] 2020-01-07 21:45:29 CodecPool: INFO: Got brand-new decompressor [.bgz] 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 32 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 37 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 33 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 44 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 47 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 30 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 48 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 40 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 41 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 29 2020-01-07 21:45:30 BlockManagerInfo: INFO: Removed broadcast_5_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 8.7 KB, free: 366.2 MB) 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 26 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 31 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 46 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 43 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 45 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 39 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 49 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 34 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 50 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 35 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 38 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 28 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 42 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 27 2020-01-07 21:45:30 ContextCleaner: INFO: Cleaned accumulator 36 2020-01-07 21:45:32 Executor: INFO: Finished task 0.0 in stage 2.0 (TID 3). 939 bytes result sent to driver 2020-01-07 21:45:32 Executor: INFO: Finished task 1.0 in stage 2.0 (TID 4). 939 bytes result sent to driver 2020-01-07 21:45:32 TaskSetManager: INFO: Finished task 1.0 in stage 2.0 (TID 4) in 2813 ms on localhost (executor driver) (1/2) 2020-01-07 21:45:32 TaskSetManager: INFO: Finished task 0.0 in stage 2.0 (TID 3) in 2816 ms on localhost (executor driver) (2/2) 2020-01-07 21:45:32 TaskSchedulerImpl: INFO: Removed TaskSet 2.0, whose tasks have all completed, from pool 2020-01-07 21:45:32 DAGScheduler: INFO: ResultStage 2 (collect at ContextRDD.scala:223) finished in 2.828 s 2020-01-07 21:45:32 DAGScheduler: INFO: Job 2 finished: collect at ContextRDD.scala:223, took 2.831482 s 2020-01-07 21:45:32 root: INFO: encoder cache miss (0 hits, 5 misses, 0.000) 2020-01-07 21:45:32 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_4. 2020-01-07 21:45:32 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_4.apply 2020-01-07 21:45:32 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_4.m0_ENCODE_o_tuple_of_END_TO_o_struct_of_END 2020-01-07 21:45:32 root: INFO: encoder cache hit 2020-01-07 21:45:32 root: INFO: encoder cache miss (1 hits, 6 misses, 0.143) 2020-01-07 21:45:32 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_5. 2020-01-07 21:45:32 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_5.apply 2020-01-07 21:45:32 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_5.m0_ENCODE_o_tuple_of_o_stringEND_TO_o_struct_of_o_binaryEND 2020-01-07 21:45:32 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_5.m1_write_fields_group_0 2020-01-07 21:45:32 root: INFO: instruction count: 16: is/hail/codegen/generated/C_etypeEncode_5.m2_ENCODE_o_binary_TO_o_binary 2020-01-07 21:45:32 Hail: INFO: wrote matrix table with 10961 rows and 284 columns in 2 partitions to data/1kg.mt 2020-01-07 21:45:32 root: INFO: took 4.044s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation : 4.045s, total 4.118s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- Verify : 0.284ms, total 4.118s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable : 4.046s, total 4.118s, tagged coverage 100.0 2020-01-07 21:45:32 root: INFO: optimize InterpretNonCompilable, post Lowering: before: IR size 1: (Begin) 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.372ms, total 4.119s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.264ms, total 4.119s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.039ms, total 4.120s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 0.097ms, total 4.120s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.036ms, total 4.120s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 0.087ms, total 4.120s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 1.659ms, total 4.120s, tagged coverage 54.0 2020-01-07 21:45:32 root: INFO: optimize InterpretNonCompilable, post Lowering: after: IR size 1: (Begin) 2020-01-07 21:45:32 root: INFO: Timer: all timings: 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 2.000ms, total 16.846ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 3.630ms, total 21.277ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 7.492ms, total 29.102ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 6.792ms, total 36.242ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 1.145ms, total 37.722ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 13.480ms, total 51.599ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 37.319ms, total 51.814ms, tagged coverage 92.6 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- Verify : 0.342ms, total 54.554ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- LoweringTransformation : 2.371ms, total 57.243ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- Verify : 0.338ms, total 57.912ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable : 4.127ms, total 58.135ms, tagged coverage 73.9 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.073ms, total 64.597ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.139ms, total 64.978ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 3.033ms, total 68.167ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 1.181ms, total 69.556ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.034ms, total 69.799ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 1.044ms, total 71.025ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 6.751ms, total 71.269ms, tagged coverage 81.5 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- Verify : 0.009ms, total 72.354ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation : 4.045s, total 4.118s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- Verify : 0.284ms, total 4.118s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable : 4.046s, total 4.118s, tagged coverage 100.0 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.372ms, total 4.119s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.264ms, total 4.119s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.039ms, total 4.120s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 0.097ms, total 4.120s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.036ms, total 4.120s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 0.087ms, total 4.120s 2020-01-07 21:45:32 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 1.659ms, total 4.120s, tagged coverage 54.0 2020-01-07 21:45:32 root: INFO: Timer: aggregate: 2020-01-07 21:45:32 root: INFO: Timer: Time taken for tag 'Verify' (4): 0.973ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for tag 'ForwardRelationalLets' (3): 1.215ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for tag 'FoldConstants' (3): 2.445ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for tag 'ExtractIntervalFilters' (3): 4.033ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for tag 'ForwardLets' (3): 8.069ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for tag 'Simplify' (3): 10.565ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for tag 'PruneDeadFields' (3): 14.612ms 2020-01-07 21:45:32 root: INFO: Timer: Time taken for tag 'LoweringTransformation' (2): 4.048s 2020-01-07 21:45:32 root: INFO: Timer: Fraction covered by a tagged leaf: 31.270ms (99.2%) 2020-01-07 21:45:34 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:45:34 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:45:58 root: INFO: globbing path data/1kg_annotations.txt returned 1 files: 1kg_annotations.txt 2020-01-07 21:45:58 MemoryStore: INFO: Block broadcast_7 stored as values in memory (estimated size 237.4 KB, free 365.4 MB) 2020-01-07 21:45:58 MemoryStore: INFO: Block broadcast_7_piece0 stored as bytes in memory (estimated size 23.0 KB, free 365.4 MB) 2020-01-07 21:45:58 BlockManagerInfo: INFO: Added broadcast_7_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 23.0 KB, free: 366.2 MB) 2020-01-07 21:45:58 SparkContext: INFO: Created broadcast 7 from textFile at RichSparkContext.scala:16 2020-01-07 21:45:58 FileInputFormat: INFO: Total input paths to process : 1 2020-01-07 21:45:58 Hail: INFO: Reading table to impute column types 2020-01-07 21:45:58 SparkContext: INFO: Starting job: reduce at TextTableReader.scala:192 2020-01-07 21:45:58 DAGScheduler: INFO: Got job 3 (reduce at TextTableReader.scala:192) with 32 output partitions 2020-01-07 21:45:58 DAGScheduler: INFO: Final stage: ResultStage 3 (reduce at TextTableReader.scala:192) 2020-01-07 21:45:58 DAGScheduler: INFO: Parents of final stage: List() 2020-01-07 21:45:58 DAGScheduler: INFO: Missing parents: List() 2020-01-07 21:45:58 DAGScheduler: INFO: Submitting ResultStage 3 (MapPartitionsRDD[16] at mapPartitions at TextTableReader.scala:165), which has no missing parents 2020-01-07 21:45:58 MemoryStore: INFO: Block broadcast_8 stored as values in memory (estimated size 6.5 KB, free 365.4 MB) 2020-01-07 21:45:58 MemoryStore: INFO: Block broadcast_8_piece0 stored as bytes in memory (estimated size 3.2 KB, free 365.4 MB) 2020-01-07 21:45:58 BlockManagerInfo: INFO: Added broadcast_8_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 3.2 KB, free: 366.2 MB) 2020-01-07 21:45:58 SparkContext: INFO: Created broadcast 8 from broadcast at DAGScheduler.scala:1161 2020-01-07 21:45:58 DAGScheduler: INFO: Submitting 32 missing tasks from ResultStage 3 (MapPartitionsRDD[16] at mapPartitions at TextTableReader.scala:165) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)) 2020-01-07 21:45:58 TaskSchedulerImpl: INFO: Adding task set 3.0 with 32 tasks 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 0.0 in stage 3.0 (TID 5, localhost, executor driver, partition 0, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 1.0 in stage 3.0 (TID 6, localhost, executor driver, partition 1, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 2.0 in stage 3.0 (TID 7, localhost, executor driver, partition 2, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 3.0 in stage 3.0 (TID 8, localhost, executor driver, partition 3, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 4.0 in stage 3.0 (TID 9, localhost, executor driver, partition 4, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 5.0 in stage 3.0 (TID 10, localhost, executor driver, partition 5, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 6.0 in stage 3.0 (TID 11, localhost, executor driver, partition 6, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 7.0 in stage 3.0 (TID 12, localhost, executor driver, partition 7, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 8.0 in stage 3.0 (TID 13, localhost, executor driver, partition 8, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 9.0 in stage 3.0 (TID 14, localhost, executor driver, partition 9, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 10.0 in stage 3.0 (TID 15, localhost, executor driver, partition 10, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 11.0 in stage 3.0 (TID 16, localhost, executor driver, partition 11, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 12.0 in stage 3.0 (TID 17, localhost, executor driver, partition 12, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 13.0 in stage 3.0 (TID 18, localhost, executor driver, partition 13, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 14.0 in stage 3.0 (TID 19, localhost, executor driver, partition 14, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 15.0 in stage 3.0 (TID 20, localhost, executor driver, partition 15, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 16.0 in stage 3.0 (TID 21, localhost, executor driver, partition 16, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 17.0 in stage 3.0 (TID 22, localhost, executor driver, partition 17, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 18.0 in stage 3.0 (TID 23, localhost, executor driver, partition 18, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 19.0 in stage 3.0 (TID 24, localhost, executor driver, partition 19, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 20.0 in stage 3.0 (TID 25, localhost, executor driver, partition 20, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 21.0 in stage 3.0 (TID 26, localhost, executor driver, partition 21, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 22.0 in stage 3.0 (TID 27, localhost, executor driver, partition 22, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 23.0 in stage 3.0 (TID 28, localhost, executor driver, partition 23, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 24.0 in stage 3.0 (TID 29, localhost, executor driver, partition 24, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 25.0 in stage 3.0 (TID 30, localhost, executor driver, partition 25, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 26.0 in stage 3.0 (TID 31, localhost, executor driver, partition 26, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 27.0 in stage 3.0 (TID 32, localhost, executor driver, partition 27, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 28.0 in stage 3.0 (TID 33, localhost, executor driver, partition 28, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 29.0 in stage 3.0 (TID 34, localhost, executor driver, partition 29, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 30.0 in stage 3.0 (TID 35, localhost, executor driver, partition 30, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 TaskSetManager: INFO: Starting task 31.0 in stage 3.0 (TID 36, localhost, executor driver, partition 31, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:45:58 Executor: INFO: Running task 1.0 in stage 3.0 (TID 6) 2020-01-07 21:45:58 Executor: INFO: Running task 0.0 in stage 3.0 (TID 5) 2020-01-07 21:45:58 Executor: INFO: Running task 2.0 in stage 3.0 (TID 7) 2020-01-07 21:45:58 Executor: INFO: Running task 3.0 in stage 3.0 (TID 8) 2020-01-07 21:45:58 Executor: INFO: Running task 4.0 in stage 3.0 (TID 9) 2020-01-07 21:45:58 Executor: INFO: Running task 5.0 in stage 3.0 (TID 10) 2020-01-07 21:45:58 Executor: INFO: Running task 6.0 in stage 3.0 (TID 11) 2020-01-07 21:45:58 Executor: INFO: Running task 7.0 in stage 3.0 (TID 12) 2020-01-07 21:45:58 Executor: INFO: Running task 8.0 in stage 3.0 (TID 13) 2020-01-07 21:45:58 Executor: INFO: Running task 9.0 in stage 3.0 (TID 14) 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:15860+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:12688+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:3172+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:0+3172 2020-01-07 21:45:58 Executor: INFO: Running task 10.0 in stage 3.0 (TID 15) 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:19032+3172 2020-01-07 21:45:58 Executor: INFO: Running task 11.0 in stage 3.0 (TID 16) 2020-01-07 21:45:58 Executor: INFO: Running task 12.0 in stage 3.0 (TID 17) 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:25376+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:22204+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:9516+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:28548+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:6344+3172 2020-01-07 21:45:58 Executor: INFO: Running task 13.0 in stage 3.0 (TID 18) 2020-01-07 21:45:58 Executor: INFO: Running task 15.0 in stage 3.0 (TID 20) 2020-01-07 21:45:58 Executor: INFO: Running task 14.0 in stage 3.0 (TID 19) 2020-01-07 21:45:58 Executor: INFO: Running task 17.0 in stage 3.0 (TID 22) 2020-01-07 21:45:58 Executor: INFO: Running task 16.0 in stage 3.0 (TID 21) 2020-01-07 21:45:58 Executor: INFO: Running task 18.0 in stage 3.0 (TID 23) 2020-01-07 21:45:58 Executor: INFO: Running task 20.0 in stage 3.0 (TID 25) 2020-01-07 21:45:58 Executor: INFO: Running task 21.0 in stage 3.0 (TID 26) 2020-01-07 21:45:58 Executor: INFO: Running task 19.0 in stage 3.0 (TID 24) 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:38064+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:44408+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:53924+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:63440+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:57096+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:34892+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:66612+3172 2020-01-07 21:45:58 Executor: INFO: Running task 22.0 in stage 3.0 (TID 27) 2020-01-07 21:45:58 Executor: INFO: Running task 24.0 in stage 3.0 (TID 29) 2020-01-07 21:45:58 Executor: INFO: Running task 26.0 in stage 3.0 (TID 31) 2020-01-07 21:45:58 Executor: INFO: Running task 27.0 in stage 3.0 (TID 32) 2020-01-07 21:45:58 Executor: INFO: Running task 23.0 in stage 3.0 (TID 28) 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:47580+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:60268+3172 2020-01-07 21:45:58 Executor: INFO: Running task 29.0 in stage 3.0 (TID 34) 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:31720+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:41236+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:50752+3172 2020-01-07 21:45:58 Executor: INFO: Running task 31.0 in stage 3.0 (TID 36) 2020-01-07 21:45:58 Executor: INFO: Running task 28.0 in stage 3.0 (TID 33) 2020-01-07 21:45:58 Executor: INFO: Running task 25.0 in stage 3.0 (TID 30) 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:69784+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:76128+3172 2020-01-07 21:45:58 Executor: INFO: Running task 30.0 in stage 3.0 (TID 35) 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:79300+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:85644+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:88816+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:95160+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:72956+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:98332+3173 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:82472+3172 2020-01-07 21:45:58 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:91988+3172 2020-01-07 21:45:58 Executor: INFO: Finished task 19.0 in stage 3.0 (TID 24). 815 bytes result sent to driver 2020-01-07 21:45:58 Executor: INFO: Finished task 23.0 in stage 3.0 (TID 28). 815 bytes result sent to driver 2020-01-07 21:45:58 Executor: INFO: Finished task 9.0 in stage 3.0 (TID 14). 858 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 23.0 in stage 3.0 (TID 28) in 60 ms on localhost (executor driver) (1/32) 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 19.0 in stage 3.0 (TID 24) in 63 ms on localhost (executor driver) (2/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 18.0 in stage 3.0 (TID 23). 858 bytes result sent to driver 2020-01-07 21:45:58 Executor: INFO: Finished task 6.0 in stage 3.0 (TID 11). 815 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 6.0 in stage 3.0 (TID 11) in 68 ms on localhost (executor driver) (3/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 13.0 in stage 3.0 (TID 18). 815 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 9.0 in stage 3.0 (TID 14) in 68 ms on localhost (executor driver) (4/32) 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 18.0 in stage 3.0 (TID 23) in 66 ms on localhost (executor driver) (5/32) 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 13.0 in stage 3.0 (TID 18) in 68 ms on localhost (executor driver) (6/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 21.0 in stage 3.0 (TID 26). 815 bytes result sent to driver 2020-01-07 21:45:58 Executor: INFO: Finished task 5.0 in stage 3.0 (TID 10). 815 bytes result sent to driver 2020-01-07 21:45:58 Executor: INFO: Finished task 4.0 in stage 3.0 (TID 9). 815 bytes result sent to driver 2020-01-07 21:45:58 Executor: INFO: Finished task 14.0 in stage 3.0 (TID 19). 815 bytes result sent to driver 2020-01-07 21:45:58 Executor: INFO: Finished task 2.0 in stage 3.0 (TID 7). 815 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 21.0 in stage 3.0 (TID 26) in 69 ms on localhost (executor driver) (7/32) 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 5.0 in stage 3.0 (TID 10) in 75 ms on localhost (executor driver) (8/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 25.0 in stage 3.0 (TID 30). 815 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 4.0 in stage 3.0 (TID 9) in 75 ms on localhost (executor driver) (9/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 12.0 in stage 3.0 (TID 17). 815 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 14.0 in stage 3.0 (TID 19) in 72 ms on localhost (executor driver) (10/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 8.0 in stage 3.0 (TID 13). 815 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 2.0 in stage 3.0 (TID 7) in 77 ms on localhost (executor driver) (11/32) 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 25.0 in stage 3.0 (TID 30) in 70 ms on localhost (executor driver) (12/32) 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 12.0 in stage 3.0 (TID 17) in 74 ms on localhost (executor driver) (13/32) 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 8.0 in stage 3.0 (TID 13) in 77 ms on localhost (executor driver) (14/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 3.0 in stage 3.0 (TID 8). 815 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 3.0 in stage 3.0 (TID 8) in 81 ms on localhost (executor driver) (15/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 20.0 in stage 3.0 (TID 25). 815 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 20.0 in stage 3.0 (TID 25) in 76 ms on localhost (executor driver) (16/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 28.0 in stage 3.0 (TID 33). 815 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 28.0 in stage 3.0 (TID 33) in 75 ms on localhost (executor driver) (17/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 7.0 in stage 3.0 (TID 12). 858 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 7.0 in stage 3.0 (TID 12) in 84 ms on localhost (executor driver) (18/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 24.0 in stage 3.0 (TID 29). 815 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 24.0 in stage 3.0 (TID 29) in 79 ms on localhost (executor driver) (19/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 15.0 in stage 3.0 (TID 20). 815 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 15.0 in stage 3.0 (TID 20) in 83 ms on localhost (executor driver) (20/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 1.0 in stage 3.0 (TID 6). 815 bytes result sent to driver 2020-01-07 21:45:58 Executor: INFO: Finished task 22.0 in stage 3.0 (TID 27). 858 bytes result sent to driver 2020-01-07 21:45:58 Executor: INFO: Finished task 30.0 in stage 3.0 (TID 35). 815 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 1.0 in stage 3.0 (TID 6) in 89 ms on localhost (executor driver) (21/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 26.0 in stage 3.0 (TID 31). 815 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 22.0 in stage 3.0 (TID 27) in 83 ms on localhost (executor driver) (22/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 17.0 in stage 3.0 (TID 22). 815 bytes result sent to driver 2020-01-07 21:45:58 Executor: INFO: Finished task 29.0 in stage 3.0 (TID 34). 815 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 30.0 in stage 3.0 (TID 35) in 81 ms on localhost (executor driver) (23/32) 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 26.0 in stage 3.0 (TID 31) in 84 ms on localhost (executor driver) (24/32) 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 17.0 in stage 3.0 (TID 22) in 86 ms on localhost (executor driver) (25/32) 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 29.0 in stage 3.0 (TID 34) in 83 ms on localhost (executor driver) (26/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 27.0 in stage 3.0 (TID 32). 858 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 27.0 in stage 3.0 (TID 32) in 86 ms on localhost (executor driver) (27/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 10.0 in stage 3.0 (TID 15). 815 bytes result sent to driver 2020-01-07 21:45:58 Executor: INFO: Finished task 16.0 in stage 3.0 (TID 21). 815 bytes result sent to driver 2020-01-07 21:45:58 Executor: INFO: Finished task 11.0 in stage 3.0 (TID 16). 815 bytes result sent to driver 2020-01-07 21:45:58 Executor: INFO: Finished task 0.0 in stage 3.0 (TID 5). 815 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 10.0 in stage 3.0 (TID 15) in 94 ms on localhost (executor driver) (28/32) 2020-01-07 21:45:58 Executor: INFO: Finished task 31.0 in stage 3.0 (TID 36). 815 bytes result sent to driver 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 16.0 in stage 3.0 (TID 21) in 92 ms on localhost (executor driver) (29/32) 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 11.0 in stage 3.0 (TID 16) in 94 ms on localhost (executor driver) (30/32) 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 0.0 in stage 3.0 (TID 5) in 99 ms on localhost (executor driver) (31/32) 2020-01-07 21:45:58 TaskSetManager: INFO: Finished task 31.0 in stage 3.0 (TID 36) in 88 ms on localhost (executor driver) (32/32) 2020-01-07 21:45:58 TaskSchedulerImpl: INFO: Removed TaskSet 3.0, whose tasks have all completed, from pool 2020-01-07 21:45:58 DAGScheduler: INFO: ResultStage 3 (reduce at TextTableReader.scala:192) finished in 0.120 s 2020-01-07 21:45:58 DAGScheduler: INFO: Job 3 finished: reduce at TextTableReader.scala:192, took 0.127151 s 2020-01-07 21:45:58 Hail: INFO: Finished type imputation Loading column 'Sample' as type 'str' (imputed) Loading column 'Population' as type 'str' (imputed) Loading column 'SuperPopulation' as type 'str' (imputed) Loading column 'isFemale' as type 'bool' (imputed) Loading column 'PurpleHair' as type 'bool' (imputed) Loading column 'CaffeineConsumption' as type 'int32' (imputed) 2020-01-07 21:46:37 root: INFO: optimize Lowerer, initial IR: before: IR size 23: (TableAggregate (MatrixRowsTable (MatrixMapCols None (MatrixAnnotateColsTable "__uid_3" (MatrixRead None False False "{\"name\":\"MatrixNativeReader\",\"path\":\"data/1kg.mt\",\"_spec\":null}") (TableKeyBy (Sample) False (TableRead None False "{\"name\":\"TextTableReader\",\"options\":{\"files\":[\"data/1kg_annotations.txt\"],\"typeMapStr\":{},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"impute\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"forceGZ\":false}}"))) (InsertFields (SelectFields (s) (Ref sa)) None (pheno (GetField __uid_3 (Ref sa)))))) (AggGroupBy False (MakeStruct (ref (ApplyIR indexArray String (GetField alleles (Ref row)) (I32 0))) (alt (ApplyIR indexArray String (GetField alleles (Ref row)) (I32 1)))) (ApplyAggOp Count () ()))) 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.347ms, total 8.035ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 2.857ms, total 11.197ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 6.177ms, total 17.572ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 6.107ms, total 23.863ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.235ms, total 24.331ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 13.326ms, total 37.827ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.404ms, total 38.526ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.127ms, total 38.799ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.528ms, total 39.453ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 1.317ms, total 40.905ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.161ms, total 41.250ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 2.295ms, total 43.671ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.443ms, total 44.333ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.130ms, total 44.608ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.309ms, total 45.061ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 0.928ms, total 46.116ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.152ms, total 46.419ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 1.726ms, total 48.266ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 40.730ms, total 48.405ms, tagged coverage 92.2 2020-01-07 21:46:37 root: INFO: optimize Lowerer, initial IR: after: IR size 15: (TableAggregate (TableKeyBy () False (MatrixRowsTable (MatrixRead Matrix{global:Struct{},col_key:[s],col:Struct{s:String},row_key:[[locus,alleles]],row:Struct{locus:Locus(GRCh37),alleles:Array[String]},entry:Struct{}} True False "{\"name\":\"MatrixNativeReader\",\"path\":\"data/1kg.mt\",\"_spec\":null}"))) (AggGroupBy False (MakeStruct (ref (ArrayRef (GetField alleles (Ref row)) (I32 0))) (alt (ArrayRef (GetField alleles (Ref row)) (I32 1)))) (ApplyAggOp Count () ()))) 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- Verify : 0.035ms, total 49.951ms 2020-01-07 21:46:37 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- LoweringTransformation : 21.967ms, total 72.061ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- Verify : 0.126ms, total 72.328ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable : 22.522ms, total 72.433ms, tagged coverage 98.3 2020-01-07 21:46:37 root: INFO: optimize LowerMatrixToTable, post Lowering: before: IR size 28: (TableAggregate (TableKeyBy () False (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String]}} False "{\"name\":\"TableNativeReader\",\"path\":\"data/1kg.mt/rows\",\"_spec\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AC:Array[Int32],AF:Array[Float64],AN:Int32,BaseQRankSum:Float64,ClippingRankSum:Float64,DP:Int32,DS:Boolean,FS:Float64,HaplotypeScore:Float64,InbreedingCoeff:Float64,MLEAC:Array[Int32],MLEAF:Array[Float64],MQ:Float64,MQ0:Int32,MQRankSum:Float64,QD:Float64,ReadPosRankSum:Float64,set:String}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}}}") (InsertFields (Ref global) None (__cols (MakeArray Array[Struct{s:String}])))) (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (MakeArray Array[Struct{}])))) (SelectFields () (Ref global))) (SelectFields (locus alleles) (Ref row)))) (AggGroupBy False (MakeStruct (ref (ArrayRef (GetField alleles (Ref row)) (I32 0))) (alt (ArrayRef (GetField alleles (Ref row)) (I32 1)))) (ApplyAggOp Count () ()))) 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 1.695ms, total 80.857ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.399ms, total 81.393ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.534ms, total 82.044ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 3.927ms, total 86.085ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.271ms, total 86.510ms 2020-01-07 21:46:37 root: INFO: Prune: InsertFields: eliminating field '__cols' 2020-01-07 21:46:37 root: INFO: Prune: InsertFields: eliminating field 'the entries! [877f12a8827e18f61222c6c8c5fb04a8]' 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 9.234ms, total 95.873ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.386ms, total 96.436ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.322ms, total 97.029ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 3.216ms, total 100.355ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 1.907ms, total 102.380ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.147ms, total 102.666ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 2.428ms, total 105.195ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.309ms, total 105.624ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.187ms, total 105.917ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.381ms, total 106.410ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 1.069ms, total 107.578ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.142ms, total 107.833ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 1.307ms, total 109.237ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 30.192ms, total 109.349ms, tagged coverage 92.3 2020-01-07 21:46:37 root: INFO: optimize LowerMatrixToTable, post Lowering: after: IR size 17: (TableAggregate (TableKeyBy () False (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String]}} False "{\"name\":\"TableNativeReader\",\"path\":\"data/1kg.mt/rows\",\"_spec\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AC:Array[Int32],AF:Array[Float64],AN:Int32,BaseQRankSum:Float64,ClippingRankSum:Float64,DP:Int32,DS:Boolean,FS:Float64,HaplotypeScore:Float64,InbreedingCoeff:Float64,MLEAC:Array[Int32],MLEAF:Array[Float64],MQ:Float64,MQ0:Int32,MQRankSum:Float64,QD:Float64,ReadPosRankSum:Float64,set:String}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}}}")) (AggLet __iruid_6 False (SelectFields (alleles) (Ref row)) (AggGroupBy False (MakeStruct (ref (ArrayRef (GetField alleles (Ref __iruid_6)) (I32 0))) (alt (ArrayRef (GetField alleles (Ref __iruid_6)) (I32 1)))) (ApplyAggOp Count () ())))) 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- Verify : 0.032ms, total 111.182ms 2020-01-07 21:46:37 root: INFO: interpreting non compilable node: TableAggregate 2020-01-07 21:46:37 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:46:37 root: INFO: decoder cache miss (0 hits, 1 misses, 0.000 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeDecode_6. 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeDecode_6.apply 2020-01-07 21:46:37 root: INFO: instruction count: 14: is/hail/codegen/generated/C_etypeDecode_6.m0_DECODE_o_struct_of_END_TO_o_tuple_of_END 2020-01-07 21:46:37 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:46:37 root: INFO: decoder cache miss (0 hits, 2 misses, 0.000 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeDecode_7. 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeDecode_7.apply 2020-01-07 21:46:37 root: INFO: instruction count: 27: is/hail/codegen/generated/C_etypeDecode_7.m0_DECODE_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryANDo_binaryANDo_float64ANDo_array_of_o_binaryANDo_struct_of_o_array_of_o_int32ANDo_array_of_o_float64ANDo_int32ANDo_float64ANDo_float64ANDo_int32ANDo_boolANDo_float64ANDo_float64ANDo_float64ANDo_array_of_o_int32ANDo_array_of_o_float64ANDo_float64ANDo_int32ANDo_float64ANDo_float64ANDo_float64ANDo_binaryENDEND_TO_o_tuple_of_o_locusANDo_array_of_o_stringEND 2020-01-07 21:46:37 root: INFO: instruction count: 109: is/hail/codegen/generated/C_etypeDecode_7.m1_read_fields_group_0 2020-01-07 21:46:37 root: INFO: instruction count: 14: is/hail/codegen/generated/C_etypeDecode_7.m2_INPLACE_DECODE_o_struct_of_r_binaryANDr_int32END_TO_o_tuple_of_r_binaryANDr_int32END 2020-01-07 21:46:37 root: INFO: instruction count: 17: is/hail/codegen/generated/C_etypeDecode_7.m3_read_fields_group_0 2020-01-07 21:46:37 root: INFO: instruction count: 29: is/hail/codegen/generated/C_etypeDecode_7.m4_INPLACE_DECODE_r_binary_TO_r_binary 2020-01-07 21:46:37 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_7.m5_INPLACE_DECODE_r_int32_TO_r_int32 2020-01-07 21:46:37 root: INFO: instruction count: 97: is/hail/codegen/generated/C_etypeDecode_7.m6_INPLACE_DECODE_o_array_of_o_binary_TO_o_array_of_o_binary 2020-01-07 21:46:37 root: INFO: instruction count: 29: is/hail/codegen/generated/C_etypeDecode_7.m7_INPLACE_DECODE_o_binary_TO_o_binary 2020-01-07 21:46:37 root: INFO: instruction count: 9: is/hail/codegen/generated/C_etypeDecode_7.m8_SKIP_o_binary 2020-01-07 21:46:37 root: INFO: instruction count: 5: is/hail/codegen/generated/C_etypeDecode_7.m9_SKIP_o_float64 2020-01-07 21:46:37 root: INFO: instruction count: 53: is/hail/codegen/generated/C_etypeDecode_7.m10_SKIP_o_array_of_o_binary 2020-01-07 21:46:37 root: INFO: instruction count: 281: is/hail/codegen/generated/C_etypeDecode_7.m11_SKIP_o_struct_of_o_array_of_o_int32ANDo_array_of_o_float64ANDo_int32ANDo_float64ANDo_float64ANDo_int32ANDo_boolANDo_float64ANDo_float64ANDo_float64ANDo_array_of_o_int32ANDo_array_of_o_float64ANDo_float64ANDo_int32ANDo_float64ANDo_float64ANDo_float64ANDo_binaryEND 2020-01-07 21:46:37 root: INFO: instruction count: 53: is/hail/codegen/generated/C_etypeDecode_7.m12_SKIP_o_array_of_o_int32 2020-01-07 21:46:37 root: INFO: instruction count: 5: is/hail/codegen/generated/C_etypeDecode_7.m13_SKIP_o_int32 2020-01-07 21:46:37 root: INFO: instruction count: 53: is/hail/codegen/generated/C_etypeDecode_7.m14_SKIP_o_array_of_o_float64 2020-01-07 21:46:37 root: INFO: instruction count: 5: is/hail/codegen/generated/C_etypeDecode_7.m15_SKIP_o_bool 2020-01-07 21:46:37 root: INFO: encoder cache hit 2020-01-07 21:46:37 root: INFO: decoder cache hit 2020-01-07 21:46:37 MemoryStore: INFO: Block broadcast_9 stored as values in memory (estimated size 1784.0 B, free 365.4 MB) 2020-01-07 21:46:37 MemoryStore: INFO: Block broadcast_9_piece0 stored as bytes in memory (estimated size 944.0 B, free 365.4 MB) 2020-01-07 21:46:37 BlockManagerInfo: INFO: Added broadcast_9_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 944.0 B, free: 366.2 MB) 2020-01-07 21:46:37 SparkContext: INFO: Created broadcast 9 from broadcast at SparkBackend.scala:21 2020-01-07 21:46:37 root: INFO: optimize Compile: before: IR size 4: (Begin (InitOp2 0 (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None))) ( (Begin (InitOp2 0 (Count () () None) ()))))) 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.084ms, total 207.634ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.069ms, total 207.984ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.064ms, total 208.131ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.360ms, total 208.681ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.041ms, total 208.804ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.152ms, total 209.019ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 1.612ms, total 209.156ms, tagged coverage 47.7 2020-01-07 21:46:37 root: INFO: optimize Compile: after: IR size 4: (Begin (InitOp2 0 (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None))) ( (Begin (InitOp2 0 (Count () () None) ()))))) 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C8. 2020-01-07 21:46:37 root: INFO: instruction count: 63: is/hail/codegen/generated/C8.apply 2020-01-07 21:46:37 root: INFO: instruction count: 14: is/hail/codegen/generated/C8.apply 2020-01-07 21:46:37 root: INFO: instruction count: 146: is/hail/codegen/generated/C8.m1_method 2020-01-07 21:46:37 root: INFO: instruction count: 105: is/hail/codegen/generated/C8.m2_method 2020-01-07 21:46:37 root: INFO: instruction count: 14: is/hail/codegen/generated/C8.m3_compWithKey 2020-01-07 21:46:37 root: INFO: instruction count: 3076: is/hail/codegen/generated/C8.m4_btree_insert 2020-01-07 21:46:37 root: INFO: instruction count: 493: is/hail/codegen/generated/C8.m5_btree_get 2020-01-07 21:46:37 root: INFO: instruction count: 1475: is/hail/codegen/generated/C8.m6_btree_deepCopy 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C8.m7_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C8.m8_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C8.m9_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C8.m10_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C8.m11_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C8.m12_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C8.m13_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C8.m14_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C8.m15_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C8.m16_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C8.m17_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C8.m18_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 21: is/hail/codegen/generated/C8.newAggState 2020-01-07 21:46:37 root: INFO: instruction count: 17: is/hail/codegen/generated/C8.setAggState 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C8.getAggOffset 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C8.setNumSerialized 2020-01-07 21:46:37 root: INFO: instruction count: 8: is/hail/codegen/generated/C8.setSerializedAgg 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C8.getSerializedAgg 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C8.m25_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 19: is/hail/codegen/generated/C8.m26_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C8.m27_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C8.m28_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 19: is/hail/codegen/generated/C8.m29_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 47: is/hail/codegen/generated/C8.m30_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 52: is/hail/codegen/generated/C8.m31_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C8.m32_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C8.m33_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 14: is/hail/codegen/generated/C8.setPartitionIndex 2020-01-07 21:46:37 root: INFO: instruction count: 6: is/hail/codegen/generated/C8.addPartitionRegion 2020-01-07 21:46:37 root: INFO: optimize Compile: before: IR size 16: (Let __iruid_6 (SelectFields (alleles) (Ref row)) (Begin (SeqOp2 0 (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None))) ( (MakeStruct (ref (ArrayRef (GetField alleles (Ref __iruid_6)) (I32 0))) (alt (ArrayRef (GetField alleles (Ref __iruid_6)) (I32 1)))) (Begin (SeqOp2 0 (Count () () None) ())))))) 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.161ms, total 286.152ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.119ms, total 286.449ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.240ms, total 286.750ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.213ms, total 288.023ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.109ms, total 288.204ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.966ms, total 289.230ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.168ms, total 289.495ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.063ms, total 289.622ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.176ms, total 289.863ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.933ms, total 290.942ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.099ms, total 291.143ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.717ms, total 291.933ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.133ms, total 292.197ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.058ms, total 292.314ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.130ms, total 292.496ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.451ms, total 294.038ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.092ms, total 294.198ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.617ms, total 294.902ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 9.001ms, total 294.985ms, tagged coverage 82.7 2020-01-07 21:46:37 root: INFO: optimize Compile: after: IR size 16: (Let __iruid_11 (SelectFields (alleles) (Ref row)) (Begin (SeqOp2 0 (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None))) ( (MakeStruct (ref (ArrayRef (GetField alleles (Ref __iruid_11)) (I32 0))) (alt (ArrayRef (GetField alleles (Ref __iruid_11)) (I32 1)))) (Begin (SeqOp2 0 (Count () () None) ())))))) 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C9. 2020-01-07 21:46:37 root: INFO: instruction count: 86: is/hail/codegen/generated/C9.apply 2020-01-07 21:46:37 root: INFO: instruction count: 20: is/hail/codegen/generated/C9.apply 2020-01-07 21:46:37 root: INFO: instruction count: 146: is/hail/codegen/generated/C9.m1_method 2020-01-07 21:46:37 root: INFO: instruction count: 105: is/hail/codegen/generated/C9.m2_method 2020-01-07 21:46:37 root: INFO: instruction count: 14: is/hail/codegen/generated/C9.m3_compWithKey 2020-01-07 21:46:37 root: INFO: instruction count: 3076: is/hail/codegen/generated/C9.m4_btree_insert 2020-01-07 21:46:37 root: INFO: instruction count: 493: is/hail/codegen/generated/C9.m5_btree_get 2020-01-07 21:46:37 root: INFO: instruction count: 1475: is/hail/codegen/generated/C9.m6_btree_deepCopy 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C9.m7_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C9.m8_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C9.m9_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C9.m10_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C9.m11_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C9.m12_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C9.m13_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C9.m14_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C9.m15_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C9.m16_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C9.m17_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C9.m18_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 21: is/hail/codegen/generated/C9.newAggState 2020-01-07 21:46:37 root: INFO: instruction count: 17: is/hail/codegen/generated/C9.setAggState 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C9.getAggOffset 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C9.setNumSerialized 2020-01-07 21:46:37 root: INFO: instruction count: 8: is/hail/codegen/generated/C9.setSerializedAgg 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C9.getSerializedAgg 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C9.m25_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 19: is/hail/codegen/generated/C9.m26_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C9.m27_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C9.m28_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 19: is/hail/codegen/generated/C9.m29_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 47: is/hail/codegen/generated/C9.m30_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 52: is/hail/codegen/generated/C9.m31_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 272: is/hail/codegen/generated/C9.m32_method 2020-01-07 21:46:37 root: INFO: instruction count: 307: is/hail/codegen/generated/C9.m33_method 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C9.m34_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C9.m35_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C9.m36_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C9.m37_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 14: is/hail/codegen/generated/C9.setPartitionIndex 2020-01-07 21:46:37 root: INFO: instruction count: 6: is/hail/codegen/generated/C9.addPartitionRegion 2020-01-07 21:46:37 root: INFO: optimize Compile: before: IR size 1: (DeserializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None))))) 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.041ms, total 330.570ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.017ms, total 330.688ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.021ms, total 330.754ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.180ms, total 330.973ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.019ms, total 331.045ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.044ms, total 331.136ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.657ms, total 331.180ms, tagged coverage 49.2 2020-01-07 21:46:37 root: INFO: optimize Compile: after: IR size 1: (DeserializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None))))) 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C10. 2020-01-07 21:46:37 root: INFO: instruction count: 37: is/hail/codegen/generated/C10.apply 2020-01-07 21:46:37 root: INFO: instruction count: 8: is/hail/codegen/generated/C10.apply 2020-01-07 21:46:37 root: INFO: instruction count: 146: is/hail/codegen/generated/C10.m1_method 2020-01-07 21:46:37 root: INFO: instruction count: 105: is/hail/codegen/generated/C10.m2_method 2020-01-07 21:46:37 root: INFO: instruction count: 14: is/hail/codegen/generated/C10.m3_compWithKey 2020-01-07 21:46:37 root: INFO: instruction count: 3076: is/hail/codegen/generated/C10.m4_btree_insert 2020-01-07 21:46:37 root: INFO: instruction count: 493: is/hail/codegen/generated/C10.m5_btree_get 2020-01-07 21:46:37 root: INFO: instruction count: 1475: is/hail/codegen/generated/C10.m6_btree_deepCopy 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C10.m7_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C10.m8_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C10.m9_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C10.m10_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C10.m11_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C10.m12_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C10.m13_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C10.m14_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C10.m15_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C10.m16_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C10.m17_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C10.m18_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 21: is/hail/codegen/generated/C10.newAggState 2020-01-07 21:46:37 root: INFO: instruction count: 17: is/hail/codegen/generated/C10.setAggState 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C10.getAggOffset 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C10.setNumSerialized 2020-01-07 21:46:37 root: INFO: instruction count: 8: is/hail/codegen/generated/C10.setSerializedAgg 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C10.getSerializedAgg 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C10.m25_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 19: is/hail/codegen/generated/C10.m26_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C10.m27_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C10.m28_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 19: is/hail/codegen/generated/C10.m29_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 47: is/hail/codegen/generated/C10.m30_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 52: is/hail/codegen/generated/C10.m31_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 27: is/hail/codegen/generated/C10.m32_DECODE_o_struct_of_o_binaryANDo_binaryEND_TO_o_tuple_of_o_stringANDo_stringEND 2020-01-07 21:46:37 root: INFO: instruction count: 53: is/hail/codegen/generated/C10.m33_read_fields_group_0 2020-01-07 21:46:37 root: INFO: instruction count: 29: is/hail/codegen/generated/C10.m34_INPLACE_DECODE_o_binary_TO_o_binary 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C10.m35_grouped_nested_deserialize_init_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C10.m36_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C10.m37_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 1639: is/hail/codegen/generated/C10.m38_btree_bulkLoad 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C10.m39_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C10.m40_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C10.m41_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C10.m42_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C10.m43_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C10.m44_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C10.m45_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C10.m46_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C10.m47_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C10.m48_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C10.m49_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C10.m50_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C10.m51_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C10.m52_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C10.m53_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C10.m54_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C10.m55_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C10.m56_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 54: is/hail/codegen/generated/C10.m57_deserialize_aggs_group0 2020-01-07 21:46:37 root: INFO: instruction count: 14: is/hail/codegen/generated/C10.setPartitionIndex 2020-01-07 21:46:37 root: INFO: instruction count: 6: is/hail/codegen/generated/C10.addPartitionRegion 2020-01-07 21:46:37 root: INFO: optimize Compile: before: IR size 1: (SerializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None))))) 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.045ms, total 357.243ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.016ms, total 357.338ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.017ms, total 357.401ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.091ms, total 357.529ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.015ms, total 357.589ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.041ms, total 357.669ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.516ms, total 357.709ms, tagged coverage 43.7 2020-01-07 21:46:37 root: INFO: optimize Compile: after: IR size 1: (SerializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None))))) 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C11. 2020-01-07 21:46:37 root: INFO: instruction count: 48: is/hail/codegen/generated/C11.apply 2020-01-07 21:46:37 root: INFO: instruction count: 8: is/hail/codegen/generated/C11.apply 2020-01-07 21:46:37 root: INFO: instruction count: 146: is/hail/codegen/generated/C11.m1_method 2020-01-07 21:46:37 root: INFO: instruction count: 105: is/hail/codegen/generated/C11.m2_method 2020-01-07 21:46:37 root: INFO: instruction count: 14: is/hail/codegen/generated/C11.m3_compWithKey 2020-01-07 21:46:37 root: INFO: instruction count: 3076: is/hail/codegen/generated/C11.m4_btree_insert 2020-01-07 21:46:37 root: INFO: instruction count: 493: is/hail/codegen/generated/C11.m5_btree_get 2020-01-07 21:46:37 root: INFO: instruction count: 1475: is/hail/codegen/generated/C11.m6_btree_deepCopy 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C11.m7_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C11.m8_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C11.m9_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C11.m10_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C11.m11_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C11.m12_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C11.m13_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C11.m14_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C11.m15_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C11.m16_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C11.m17_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C11.m18_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 21: is/hail/codegen/generated/C11.newAggState 2020-01-07 21:46:37 root: INFO: instruction count: 17: is/hail/codegen/generated/C11.setAggState 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C11.getAggOffset 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C11.setNumSerialized 2020-01-07 21:46:37 root: INFO: instruction count: 8: is/hail/codegen/generated/C11.setSerializedAgg 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C11.getSerializedAgg 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C11.m25_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 19: is/hail/codegen/generated/C11.m26_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C11.m27_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C11.m28_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 19: is/hail/codegen/generated/C11.m29_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 47: is/hail/codegen/generated/C11.m30_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 52: is/hail/codegen/generated/C11.m31_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 11: is/hail/codegen/generated/C11.m32_ENCODE_o_tuple_of_o_stringANDo_stringEND_TO_o_struct_of_o_binaryANDo_binaryEND 2020-01-07 21:46:37 root: INFO: instruction count: 39: is/hail/codegen/generated/C11.m33_write_fields_group_0 2020-01-07 21:46:37 root: INFO: instruction count: 16: is/hail/codegen/generated/C11.m34_ENCODE_o_binary_TO_o_binary 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C11.m35_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C11.m36_grouped_nested_serialize_init_group0 2020-01-07 21:46:37 root: INFO: instruction count: 513: is/hail/codegen/generated/C11.m37_btree_bulkStore 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C11.m38_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C11.m39_grouped_nested_serialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C11.m40_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C11.m41_grouped_nested_serialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C11.m42_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C11.m43_grouped_nested_serialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C11.m44_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C11.m45_grouped_nested_serialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C11.m46_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C11.m47_grouped_nested_serialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C11.m48_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C11.m49_grouped_nested_serialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C11.m50_serialize_aggs_group0 2020-01-07 21:46:37 root: INFO: instruction count: 52: is/hail/codegen/generated/C11.m51_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 14: is/hail/codegen/generated/C11.setPartitionIndex 2020-01-07 21:46:37 root: INFO: instruction count: 6: is/hail/codegen/generated/C11.addPartitionRegion 2020-01-07 21:46:37 root: INFO: optimize Compile: before: IR size 5: (Begin (DeserializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None))))) (DeserializeAggs 1 1 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None))))) (CombOp2 0 1 (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None)))) (SerializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None)))))) 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.074ms, total 383.766ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.037ms, total 383.883ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.075ms, total 384.007ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.345ms, total 384.402ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.030ms, total 384.488ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.120ms, total 384.652ms 2020-01-07 21:46:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 1.023ms, total 384.709ms, tagged coverage 66.5 2020-01-07 21:46:37 root: INFO: optimize Compile: after: IR size 5: (Begin (DeserializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None))))) (DeserializeAggs 1 1 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None))))) (CombOp2 0 1 (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None)))) (SerializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None)))))) 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12. 2020-01-07 21:46:37 root: INFO: instruction count: 96: is/hail/codegen/generated/C12.apply 2020-01-07 21:46:37 root: INFO: instruction count: 8: is/hail/codegen/generated/C12.apply 2020-01-07 21:46:37 root: INFO: instruction count: 146: is/hail/codegen/generated/C12.m1_method 2020-01-07 21:46:37 root: INFO: instruction count: 105: is/hail/codegen/generated/C12.m2_method 2020-01-07 21:46:37 root: INFO: instruction count: 14: is/hail/codegen/generated/C12.m3_compWithKey 2020-01-07 21:46:37 root: INFO: instruction count: 3076: is/hail/codegen/generated/C12.m4_btree_insert 2020-01-07 21:46:37 root: INFO: instruction count: 493: is/hail/codegen/generated/C12.m5_btree_get 2020-01-07 21:46:37 root: INFO: instruction count: 1475: is/hail/codegen/generated/C12.m6_btree_deepCopy 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m7_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m8_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m9_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m10_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m11_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m12_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m13_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m14_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m15_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m16_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m17_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m18_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 14: is/hail/codegen/generated/C12.m19_compWithKey 2020-01-07 21:46:37 root: INFO: instruction count: 3076: is/hail/codegen/generated/C12.m20_btree_insert 2020-01-07 21:46:37 root: INFO: instruction count: 493: is/hail/codegen/generated/C12.m21_btree_get 2020-01-07 21:46:37 root: INFO: instruction count: 1475: is/hail/codegen/generated/C12.m22_btree_deepCopy 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m23_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m24_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m25_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m26_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m27_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m28_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m29_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m30_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m31_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m32_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m33_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m34_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 21: is/hail/codegen/generated/C12.newAggState 2020-01-07 21:46:37 root: INFO: instruction count: 17: is/hail/codegen/generated/C12.setAggState 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.getAggOffset 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.setNumSerialized 2020-01-07 21:46:37 root: INFO: instruction count: 8: is/hail/codegen/generated/C12.setSerializedAgg 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.getSerializedAgg 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m41_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m42_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 35: is/hail/codegen/generated/C12.m43_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 11: is/hail/codegen/generated/C12.m44_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m45_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m46_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 35: is/hail/codegen/generated/C12.m47_create_states_group0 2020-01-07 21:46:37 root: INFO: instruction count: 91: is/hail/codegen/generated/C12.m48_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 101: is/hail/codegen/generated/C12.m49_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 27: is/hail/codegen/generated/C12.m50_DECODE_o_struct_of_o_binaryANDo_binaryEND_TO_o_tuple_of_o_stringANDo_stringEND 2020-01-07 21:46:37 root: INFO: instruction count: 53: is/hail/codegen/generated/C12.m51_read_fields_group_0 2020-01-07 21:46:37 root: INFO: instruction count: 29: is/hail/codegen/generated/C12.m52_INPLACE_DECODE_o_binary_TO_o_binary 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m53_grouped_nested_deserialize_init_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m54_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m55_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 1639: is/hail/codegen/generated/C12.m56_btree_bulkLoad 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m57_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m58_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m59_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m60_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m61_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m62_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m63_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m64_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m65_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m66_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m67_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m68_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m69_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m70_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m71_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m72_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m73_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m74_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 54: is/hail/codegen/generated/C12.m75_deserialize_aggs_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m76_grouped_nested_deserialize_init_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m77_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m78_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 1639: is/hail/codegen/generated/C12.m79_btree_bulkLoad 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m80_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m81_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m82_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m83_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m84_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m85_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m86_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m87_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m88_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m89_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m90_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m91_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m92_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m93_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m94_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m95_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m96_grouped_nested_deserialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m97_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 54: is/hail/codegen/generated/C12.m98_deserialize_aggs_group0 2020-01-07 21:46:37 root: INFO: instruction count: 10: is/hail/codegen/generated/C12.m99_grouped_nested_comb_group0 2020-01-07 21:46:37 root: INFO: instruction count: 1632: is/hail/codegen/generated/C12.m100_btree_foreach 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m101_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m102_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m103_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m104_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m105_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m106_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m107_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m108_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m109_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m110_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m111_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m112_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m113_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m114_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m115_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m116_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m117_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m118_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m119_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m120_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m121_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m122_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m123_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m124_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m125_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m126_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 3: is/hail/codegen/generated/C12.m127_new_state_group0 2020-01-07 21:46:37 root: INFO: instruction count: 12: is/hail/codegen/generated/C12.m128_copy_from_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m129_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m130_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 11: is/hail/codegen/generated/C12.m131_ENCODE_o_tuple_of_o_stringANDo_stringEND_TO_o_struct_of_o_binaryANDo_binaryEND 2020-01-07 21:46:37 root: INFO: instruction count: 39: is/hail/codegen/generated/C12.m132_write_fields_group_0 2020-01-07 21:46:37 root: INFO: instruction count: 16: is/hail/codegen/generated/C12.m133_ENCODE_o_binary_TO_o_binary 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m134_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m135_grouped_nested_serialize_init_group0 2020-01-07 21:46:37 root: INFO: instruction count: 513: is/hail/codegen/generated/C12.m136_btree_bulkStore 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m137_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m138_grouped_nested_serialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m139_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m140_grouped_nested_serialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m141_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m142_grouped_nested_serialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m143_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m144_grouped_nested_serialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m145_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m146_grouped_nested_serialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m147_load_group0 2020-01-07 21:46:37 root: INFO: instruction count: 7: is/hail/codegen/generated/C12.m148_grouped_nested_serialize_group0 2020-01-07 21:46:37 root: INFO: instruction count: 15: is/hail/codegen/generated/C12.m149_serialize_aggs_group0 2020-01-07 21:46:37 root: INFO: instruction count: 101: is/hail/codegen/generated/C12.m150_store_group0 2020-01-07 21:46:37 root: INFO: instruction count: 14: is/hail/codegen/generated/C12.setPartitionIndex 2020-01-07 21:46:37 root: INFO: instruction count: 6: is/hail/codegen/generated/C12.addPartitionRegion 2020-01-07 21:46:38 root: INFO: optimize Compile: before: IR size 14: (Let __iruid_7 (ResultOp2 0 ( (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None))))) (MakeTuple (0) (ToDict (ArrayMap __iruid_8 (ToArray (GetTupleElement 0 (Ref __iruid_7))) (MakeTuple (0 1) (GetField key (Ref __iruid_8)) (GetTupleElement 0 (GetField value (Ref __iruid_8)))))))) 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 3.043ms, total 442.115ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.227ms, total 442.462ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.307ms, total 442.836ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 3.088ms, total 445.990ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.099ms, total 446.185ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 3.934ms, total 450.188ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.298ms, total 450.581ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.056ms, total 450.699ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.242ms, total 451.010ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.848ms, total 451.920ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.057ms, total 452.090ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.641ms, total 452.790ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.272ms, total 453.127ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.057ms, total 453.259ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.119ms, total 453.438ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.946ms, total 454.445ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.058ms, total 454.574ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.872ms, total 455.496ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 16.488ms, total 455.551ms, tagged coverage 92.0 2020-01-07 21:46:38 root: INFO: optimize Compile: after: IR size 14: (Let __iruid_16 (ResultOp2 0 ( (Group (+Void) (Struct{ref:String,alt:String} +Void) ( (Count () () None))))) (MakeTuple (0) (ToDict (ArrayMap __iruid_17 (ToArray (GetTupleElement 0 (Ref __iruid_16))) (MakeTuple (0 1) (GetField key (Ref __iruid_17)) (GetTupleElement 0 (GetField value (Ref __iruid_17)))))))) 2020-01-07 21:46:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C14. 2020-01-07 21:46:38 root: INFO: instruction count: 11: is/hail/codegen/generated/C14.apply 2020-01-07 21:46:38 root: INFO: instruction count: 14: is/hail/codegen/generated/C14.apply 2020-01-07 21:46:38 root: INFO: instruction count: 110: is/hail/codegen/generated/C14.m1_method 2020-01-07 21:46:38 root: INFO: instruction count: 33: is/hail/codegen/generated/C14.m2_method 2020-01-07 21:46:38 root: INFO: instruction count: 146: is/hail/codegen/generated/C14.m3_method 2020-01-07 21:46:38 root: INFO: instruction count: 105: is/hail/codegen/generated/C14.m4_method 2020-01-07 21:46:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C13. 2020-01-07 21:46:38 root: INFO: instruction count: 84: is/hail/codegen/generated/C13.apply 2020-01-07 21:46:38 root: INFO: instruction count: 16: is/hail/codegen/generated/C13.apply 2020-01-07 21:46:38 root: INFO: instruction count: 146: is/hail/codegen/generated/C13.m1_method 2020-01-07 21:46:38 root: INFO: instruction count: 105: is/hail/codegen/generated/C13.m2_method 2020-01-07 21:46:38 root: INFO: instruction count: 14: is/hail/codegen/generated/C13.m3_compWithKey 2020-01-07 21:46:38 root: INFO: instruction count: 3076: is/hail/codegen/generated/C13.m4_btree_insert 2020-01-07 21:46:38 root: INFO: instruction count: 493: is/hail/codegen/generated/C13.m5_btree_get 2020-01-07 21:46:38 root: INFO: instruction count: 1475: is/hail/codegen/generated/C13.m6_btree_deepCopy 2020-01-07 21:46:38 root: INFO: instruction count: 12: is/hail/codegen/generated/C13.m7_copy_from_group0 2020-01-07 21:46:38 root: INFO: instruction count: 15: is/hail/codegen/generated/C13.m8_store_group0 2020-01-07 21:46:38 root: INFO: instruction count: 12: is/hail/codegen/generated/C13.m9_copy_from_group0 2020-01-07 21:46:38 root: INFO: instruction count: 15: is/hail/codegen/generated/C13.m10_store_group0 2020-01-07 21:46:38 root: INFO: instruction count: 12: is/hail/codegen/generated/C13.m11_copy_from_group0 2020-01-07 21:46:38 root: INFO: instruction count: 15: is/hail/codegen/generated/C13.m12_store_group0 2020-01-07 21:46:38 root: INFO: instruction count: 12: is/hail/codegen/generated/C13.m13_copy_from_group0 2020-01-07 21:46:38 root: INFO: instruction count: 15: is/hail/codegen/generated/C13.m14_store_group0 2020-01-07 21:46:38 root: INFO: instruction count: 12: is/hail/codegen/generated/C13.m15_copy_from_group0 2020-01-07 21:46:38 root: INFO: instruction count: 15: is/hail/codegen/generated/C13.m16_store_group0 2020-01-07 21:46:38 root: INFO: instruction count: 12: is/hail/codegen/generated/C13.m17_copy_from_group0 2020-01-07 21:46:38 root: INFO: instruction count: 15: is/hail/codegen/generated/C13.m18_store_group0 2020-01-07 21:46:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C13.newAggState 2020-01-07 21:46:38 root: INFO: instruction count: 17: is/hail/codegen/generated/C13.setAggState 2020-01-07 21:46:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C13.getAggOffset 2020-01-07 21:46:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C13.setNumSerialized 2020-01-07 21:46:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C13.setSerializedAgg 2020-01-07 21:46:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C13.getSerializedAgg 2020-01-07 21:46:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C13.m25_create_states_group0 2020-01-07 21:46:38 root: INFO: instruction count: 19: is/hail/codegen/generated/C13.m26_create_states_group0 2020-01-07 21:46:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C13.m27_new_state_group0 2020-01-07 21:46:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C13.m28_create_states_group0 2020-01-07 21:46:38 root: INFO: instruction count: 19: is/hail/codegen/generated/C13.m29_create_states_group0 2020-01-07 21:46:38 root: INFO: instruction count: 47: is/hail/codegen/generated/C13.m30_load_group0 2020-01-07 21:46:38 root: INFO: instruction count: 52: is/hail/codegen/generated/C13.m31_store_group0 2020-01-07 21:46:38 root: INFO: instruction count: 1392: is/hail/codegen/generated/C13.m32_btree_foreach 2020-01-07 21:46:38 root: INFO: instruction count: 15: is/hail/codegen/generated/C13.m33_load_group0 2020-01-07 21:46:38 root: INFO: instruction count: 9: is/hail/codegen/generated/C13.m34_grouped_result_group0 2020-01-07 21:46:38 root: INFO: instruction count: 15: is/hail/codegen/generated/C13.m35_load_group0 2020-01-07 21:46:38 root: INFO: instruction count: 9: is/hail/codegen/generated/C13.m36_grouped_result_group0 2020-01-07 21:46:38 root: INFO: instruction count: 15: is/hail/codegen/generated/C13.m37_load_group0 2020-01-07 21:46:38 root: INFO: instruction count: 9: is/hail/codegen/generated/C13.m38_grouped_result_group0 2020-01-07 21:46:38 root: INFO: instruction count: 15: is/hail/codegen/generated/C13.m39_load_group0 2020-01-07 21:46:38 root: INFO: instruction count: 9: is/hail/codegen/generated/C13.m40_grouped_result_group0 2020-01-07 21:46:38 root: INFO: instruction count: 15: is/hail/codegen/generated/C13.m41_load_group0 2020-01-07 21:46:38 root: INFO: instruction count: 9: is/hail/codegen/generated/C13.m42_grouped_result_group0 2020-01-07 21:46:38 root: INFO: instruction count: 15: is/hail/codegen/generated/C13.m43_load_group0 2020-01-07 21:46:38 root: INFO: instruction count: 9: is/hail/codegen/generated/C13.m44_grouped_result_group0 2020-01-07 21:46:38 root: INFO: instruction count: 41: is/hail/codegen/generated/C13.m45_aggs_result_group0 2020-01-07 21:46:38 root: INFO: instruction count: 52: is/hail/codegen/generated/C13.m46_store_group0 2020-01-07 21:46:38 root: INFO: instruction count: 893: is/hail/codegen/generated/C13.m47_method 2020-01-07 21:46:38 root: INFO: instruction count: 99: is/hail/codegen/generated/C13.m48_method 2020-01-07 21:46:38 root: INFO: instruction count: 130: is/hail/codegen/generated/C13.m49_method 2020-01-07 21:46:38 root: INFO: instruction count: 106: is/hail/codegen/generated/C13.m50_method 2020-01-07 21:46:38 root: INFO: instruction count: 121: is/hail/codegen/generated/C13.m51_method 2020-01-07 21:46:38 root: INFO: instruction count: 14: is/hail/codegen/generated/C13.setPartitionIndex 2020-01-07 21:46:38 root: INFO: instruction count: 6: is/hail/codegen/generated/C13.addPartitionRegion 2020-01-07 21:46:38 root: INFO: Aggregate: useTreeAggregate=false 2020-01-07 21:46:38 root: INFO: Aggregate: commutative=true 2020-01-07 21:46:38 SparkContext: INFO: Starting job: runJob at RVD.scala:605 2020-01-07 21:46:38 DAGScheduler: INFO: Got job 4 (runJob at RVD.scala:605) with 2 output partitions 2020-01-07 21:46:38 DAGScheduler: INFO: Final stage: ResultStage 4 (runJob at RVD.scala:605) 2020-01-07 21:46:38 DAGScheduler: INFO: Parents of final stage: List() 2020-01-07 21:46:38 DAGScheduler: INFO: Missing parents: List() 2020-01-07 21:46:38 DAGScheduler: INFO: Submitting ResultStage 4 (MapPartitionsRDD[21] at mapPartitions at ContextRDD.scala:217), which has no missing parents 2020-01-07 21:46:38 MemoryStore: INFO: Block broadcast_10 stored as values in memory (estimated size 93.0 KB, free 365.3 MB) 2020-01-07 21:46:38 MemoryStore: INFO: Block broadcast_10_piece0 stored as bytes in memory (estimated size 36.7 KB, free 365.2 MB) 2020-01-07 21:46:38 BlockManagerInfo: INFO: Added broadcast_10_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 36.7 KB, free: 366.2 MB) 2020-01-07 21:46:38 SparkContext: INFO: Created broadcast 10 from broadcast at DAGScheduler.scala:1161 2020-01-07 21:46:38 DAGScheduler: INFO: Submitting 2 missing tasks from ResultStage 4 (MapPartitionsRDD[21] at mapPartitions at ContextRDD.scala:217) (first 15 tasks are for partitions Vector(0, 1)) 2020-01-07 21:46:38 TaskSchedulerImpl: INFO: Adding task set 4.0 with 2 tasks 2020-01-07 21:46:38 TaskSetManager: INFO: Starting task 0.0 in stage 4.0 (TID 37, localhost, executor driver, partition 0, PROCESS_LOCAL, 7696 bytes) 2020-01-07 21:46:38 TaskSetManager: INFO: Starting task 1.0 in stage 4.0 (TID 38, localhost, executor driver, partition 1, PROCESS_LOCAL, 7696 bytes) 2020-01-07 21:46:38 Executor: INFO: Running task 0.0 in stage 4.0 (TID 37) 2020-01-07 21:46:38 Executor: INFO: Running task 1.0 in stage 4.0 (TID 38) 2020-01-07 21:46:38 root: INFO: RegionPool: initialized for thread 193: Executor task launch worker for task 37 2020-01-07 21:46:38 Executor: INFO: Finished task 0.0 in stage 4.0 (TID 37). 1030 bytes result sent to driver 2020-01-07 21:46:38 Executor: INFO: Finished task 1.0 in stage 4.0 (TID 38). 1030 bytes result sent to driver 2020-01-07 21:46:38 TaskSetManager: INFO: Finished task 0.0 in stage 4.0 (TID 37) in 94 ms on localhost (executor driver) (1/2) 2020-01-07 21:46:38 TaskSetManager: INFO: Finished task 1.0 in stage 4.0 (TID 38) in 92 ms on localhost (executor driver) (2/2) 2020-01-07 21:46:38 TaskSchedulerImpl: INFO: Removed TaskSet 4.0, whose tasks have all completed, from pool 2020-01-07 21:46:38 root: INFO: RegionPool: initialized for thread 87: dag-scheduler-event-loop 2020-01-07 21:46:38 DAGScheduler: INFO: ResultStage 4 (runJob at RVD.scala:605) finished in 0.116 s 2020-01-07 21:46:38 DAGScheduler: INFO: Job 4 finished: runJob at RVD.scala:605, took 0.120116 s 2020-01-07 21:46:38 root: INFO: took 558.214ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation : 558.363ms, total 669.672ms, tagged coverage 4.4 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- Verify : 0.022ms, total 669.783ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable : 558.674ms, total 669.822ms, tagged coverage 4.4 2020-01-07 21:46:38 root: INFO: optimize InterpretNonCompilable, post Lowering: before: IR size 1: (Literal Dict[Struct{ref:String,alt:String},Int64] ) 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.053ms, total 670.139ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.071ms, total 670.270ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.036ms, total 670.355ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 0.125ms, total 670.519ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.019ms, total 670.578ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 0.063ms, total 670.678ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 0.653ms, total 670.733ms, tagged coverage 56.2 2020-01-07 21:46:38 root: INFO: optimize InterpretNonCompilable, post Lowering: after: IR size 1: (Literal Dict[Struct{ref:String,alt:String},Int64] ) 2020-01-07 21:46:38 root: INFO: encoder cache miss (2 hits, 7 misses, 0.222) 2020-01-07 21:46:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_16. 2020-01-07 21:46:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_16.apply 2020-01-07 21:46:38 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_16.m0_ENCODE_o_tuple_of_o_dict_of_o_tuple_of_o_stringANDo_stringENDANDo_int64END_TO_o_struct_of_o_array_of_r_struct_of_o_struct_of_o_binaryANDo_binaryENDANDo_int64ENDEND 2020-01-07 21:46:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_16.m1_write_fields_group_0 2020-01-07 21:46:38 root: INFO: instruction count: 47: is/hail/codegen/generated/C_etypeEncode_16.m2_ENCODE_o_array_of_r_tuple_of_o_tuple_of_o_binaryANDo_binaryENDANDo_int64END_TO_o_array_of_r_struct_of_o_struct_of_o_binaryANDo_binaryENDANDo_int64END 2020-01-07 21:46:38 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_16.m3_ENCODE_r_tuple_of_o_tuple_of_o_binaryANDo_binaryENDANDo_int64END_TO_r_struct_of_o_struct_of_o_binaryANDo_binaryENDANDo_int64END 2020-01-07 21:46:38 root: INFO: instruction count: 37: is/hail/codegen/generated/C_etypeEncode_16.m4_write_fields_group_0 2020-01-07 21:46:38 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_16.m5_ENCODE_o_tuple_of_o_binaryANDo_binaryEND_TO_o_struct_of_o_binaryANDo_binaryEND 2020-01-07 21:46:38 root: INFO: instruction count: 39: is/hail/codegen/generated/C_etypeEncode_16.m6_write_fields_group_0 2020-01-07 21:46:38 root: INFO: instruction count: 16: is/hail/codegen/generated/C_etypeEncode_16.m7_ENCODE_o_binary_TO_o_binary 2020-01-07 21:46:38 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_16.m8_ENCODE_o_int64_TO_o_int64 2020-01-07 21:46:38 MemoryStore: INFO: Block broadcast_11 stored as values in memory (estimated size 296.0 B, free 365.2 MB) 2020-01-07 21:46:38 MemoryStore: INFO: Block broadcast_11_piece0 stored as bytes in memory (estimated size 213.0 B, free 365.2 MB) 2020-01-07 21:46:38 BlockManagerInfo: INFO: Added broadcast_11_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 213.0 B, free: 366.2 MB) 2020-01-07 21:46:38 SparkContext: INFO: Created broadcast 11 from broadcast at SparkBackend.scala:21 2020-01-07 21:46:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C15. 2020-01-07 21:46:38 root: INFO: instruction count: 54: is/hail/codegen/generated/C15.apply 2020-01-07 21:46:38 root: INFO: instruction count: 10: is/hail/codegen/generated/C15.apply 2020-01-07 21:46:38 root: INFO: instruction count: 14: is/hail/codegen/generated/C15.setPartitionIndex 2020-01-07 21:46:38 root: INFO: instruction count: 6: is/hail/codegen/generated/C15.addPartitionRegion 2020-01-07 21:46:38 root: INFO: instruction count: 27: is/hail/codegen/generated/C15.m3_DECODE_o_struct_of_o_array_of_r_struct_of_o_struct_of_o_binaryANDo_binaryENDANDo_int64ENDEND_TO_o_tuple_of_o_dict_of_o_tuple_of_o_stringANDo_stringENDANDo_int64END 2020-01-07 21:46:38 root: INFO: instruction count: 28: is/hail/codegen/generated/C15.m4_read_fields_group_0 2020-01-07 21:46:38 root: INFO: instruction count: 51: is/hail/codegen/generated/C15.m5_INPLACE_DECODE_o_array_of_r_struct_of_o_struct_of_o_binaryANDo_binaryENDANDo_int64END_TO_o_array_of_r_tuple_of_o_tuple_of_o_binaryANDo_binaryENDANDo_int64END 2020-01-07 21:46:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C15.m6_INPLACE_DECODE_r_struct_of_o_struct_of_o_binaryANDo_binaryENDANDo_int64END_TO_r_tuple_of_o_tuple_of_o_binaryANDo_binaryENDANDo_int64END 2020-01-07 21:46:38 root: INFO: instruction count: 53: is/hail/codegen/generated/C15.m7_read_fields_group_0 2020-01-07 21:46:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C15.m8_INPLACE_DECODE_o_struct_of_o_binaryANDo_binaryEND_TO_o_tuple_of_o_binaryANDo_binaryEND 2020-01-07 21:46:38 root: INFO: instruction count: 53: is/hail/codegen/generated/C15.m9_read_fields_group_0 2020-01-07 21:46:38 root: INFO: instruction count: 29: is/hail/codegen/generated/C15.m10_INPLACE_DECODE_o_binary_TO_o_binary 2020-01-07 21:46:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C15.m11_INPLACE_DECODE_o_int64_TO_o_int64 2020-01-07 21:46:38 root: INFO: instruction count: 30: is/hail/codegen/generated/C15.addLiterals 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Compile : 18.866ms, total 690.313ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InitializeCompiledFunction : 0.897ms, total 691.575ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- RunCompiledFunction : 0.046ms, total 691.909ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- ConvertToSafeValue : 0.934ms, total 693.118ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate : 693.172ms, total 693.180ms, tagged coverage 19.2 2020-01-07 21:46:38 root: INFO: Timer: all timings: 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.347ms, total 8.035ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 2.857ms, total 11.197ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 6.177ms, total 17.572ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 6.107ms, total 23.863ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.235ms, total 24.331ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 13.326ms, total 37.827ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.404ms, total 38.526ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.127ms, total 38.799ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.528ms, total 39.453ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 1.317ms, total 40.905ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.161ms, total 41.250ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 2.295ms, total 43.671ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.443ms, total 44.333ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.130ms, total 44.608ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.309ms, total 45.061ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 0.928ms, total 46.116ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.152ms, total 46.419ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 1.726ms, total 48.266ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 40.730ms, total 48.405ms, tagged coverage 92.2 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- Verify : 0.035ms, total 49.951ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- LoweringTransformation : 21.967ms, total 72.061ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- Verify : 0.126ms, total 72.328ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable : 22.522ms, total 72.433ms, tagged coverage 98.3 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 1.695ms, total 80.857ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.399ms, total 81.393ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.534ms, total 82.044ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 3.927ms, total 86.085ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.271ms, total 86.510ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 9.234ms, total 95.873ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.386ms, total 96.436ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.322ms, total 97.029ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 3.216ms, total 100.355ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 1.907ms, total 102.380ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.147ms, total 102.666ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 2.428ms, total 105.195ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.309ms, total 105.624ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.187ms, total 105.917ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.381ms, total 106.410ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 1.069ms, total 107.578ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.142ms, total 107.833ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 1.307ms, total 109.237ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 30.192ms, total 109.349ms, tagged coverage 92.3 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- Verify : 0.032ms, total 111.182ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.084ms, total 207.634ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.069ms, total 207.984ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.064ms, total 208.131ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.360ms, total 208.681ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.041ms, total 208.804ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.152ms, total 209.019ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 1.612ms, total 209.156ms, tagged coverage 47.7 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.161ms, total 286.152ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.119ms, total 286.449ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.240ms, total 286.750ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.213ms, total 288.023ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.109ms, total 288.204ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.966ms, total 289.230ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.168ms, total 289.495ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.063ms, total 289.622ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.176ms, total 289.863ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.933ms, total 290.942ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.099ms, total 291.143ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.717ms, total 291.933ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.133ms, total 292.197ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.058ms, total 292.314ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.130ms, total 292.496ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.451ms, total 294.038ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.092ms, total 294.198ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.617ms, total 294.902ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 9.001ms, total 294.985ms, tagged coverage 82.7 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.041ms, total 330.570ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.017ms, total 330.688ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.021ms, total 330.754ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.180ms, total 330.973ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.019ms, total 331.045ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.044ms, total 331.136ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.657ms, total 331.180ms, tagged coverage 49.2 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.045ms, total 357.243ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.016ms, total 357.338ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.017ms, total 357.401ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.091ms, total 357.529ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.015ms, total 357.589ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.041ms, total 357.669ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.516ms, total 357.709ms, tagged coverage 43.7 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.074ms, total 383.766ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.037ms, total 383.883ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.075ms, total 384.007ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.345ms, total 384.402ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.030ms, total 384.488ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.120ms, total 384.652ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 1.023ms, total 384.709ms, tagged coverage 66.5 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 3.043ms, total 442.115ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.227ms, total 442.462ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.307ms, total 442.836ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 3.088ms, total 445.990ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.099ms, total 446.185ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 3.934ms, total 450.188ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.298ms, total 450.581ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.056ms, total 450.699ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.242ms, total 451.010ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.848ms, total 451.920ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.057ms, total 452.090ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.641ms, total 452.790ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.272ms, total 453.127ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.057ms, total 453.259ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.119ms, total 453.438ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.946ms, total 454.445ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.058ms, total 454.574ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.872ms, total 455.496ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 16.488ms, total 455.551ms, tagged coverage 92.0 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation : 558.363ms, total 669.672ms, tagged coverage 4.4 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- Verify : 0.022ms, total 669.783ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable : 558.674ms, total 669.822ms, tagged coverage 4.4 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.053ms, total 670.139ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.071ms, total 670.270ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.036ms, total 670.355ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 0.125ms, total 670.519ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.019ms, total 670.578ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 0.063ms, total 670.678ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 0.653ms, total 670.733ms, tagged coverage 56.2 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Compile : 18.866ms, total 690.313ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InitializeCompiledFunction : 0.897ms, total 691.575ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- RunCompiledFunction : 0.046ms, total 691.909ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- ConvertToSafeValue : 0.934ms, total 693.118ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for CompileAndEvaluate : 693.172ms, total 693.180ms, tagged coverage 19.2 2020-01-07 21:46:38 root: INFO: Timer: aggregate: 2020-01-07 21:46:38 root: INFO: Timer: Time taken for tag 'RunCompiledFunction' (1): 0.046ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for tag 'Verify' (4): 0.215ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for tag 'InitializeCompiledFunction' (1): 0.897ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for tag 'ConvertToSafeValue' (1): 0.934ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for tag 'ForwardRelationalLets' (17): 1.745ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for tag 'ExtractIntervalFilters' (17): 4.812ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for tag 'FoldConstants' (17): 7.957ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for tag 'Simplify' (17): 12.573ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for tag 'Compile' (1): 18.866ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for tag 'LoweringTransformation' (1): 21.967ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for tag 'ForwardLets' (17): 24.835ms 2020-01-07 21:46:38 root: INFO: Timer: Time taken for tag 'PruneDeadFields' (17): 38.483ms 2020-01-07 21:46:38 root: INFO: Timer: Fraction covered by a tagged leaf: 560.563ms (19.2%) 2020-01-07 21:47:37 root: INFO: RegionPool: FREE: 1.6M allocated (1.6M blocks / 0 chunks), thread 97: Executor task launch worker for task 1 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 86 2020-01-07 21:47:37 BlockManagerInfo: INFO: Removed broadcast_7_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 23.0 KB, free: 366.2 MB) 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 83 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 99 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 92 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 62 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 60 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 73 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 110 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 120 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 69 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 70 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 80 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 121 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 96 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 101 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 88 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 107 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 53 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 104 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 64 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 72 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 67 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 103 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 122 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 125 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 81 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 114 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 95 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 113 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 52 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 75 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 58 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 119 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 105 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 54 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 98 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 116 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 63 2020-01-07 21:47:37 BlockManagerInfo: INFO: Removed broadcast_6_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 25.2 KB, free: 366.2 MB) 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 87 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 90 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 56 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 77 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 94 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 106 2020-01-07 21:47:37 BlockManagerInfo: INFO: Removed broadcast_9_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 944.0 B, free: 366.2 MB) 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 109 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 85 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 51 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 89 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 65 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 118 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 97 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 108 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 59 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 112 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 84 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 100 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 74 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 117 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 57 2020-01-07 21:47:37 BlockManagerInfo: INFO: Removed broadcast_8_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 3.2 KB, free: 366.2 MB) 2020-01-07 21:47:37 BlockManagerInfo: INFO: Removed broadcast_10_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 36.7 KB, free: 366.2 MB) 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 82 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 71 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 123 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 78 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 102 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 55 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 68 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 124 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 79 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 91 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 111 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 115 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 61 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 66 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 76 2020-01-07 21:47:37 ContextCleaner: INFO: Cleaned accumulator 93 2020-01-07 21:47:37 root: INFO: optimize Lowerer, initial IR: before: IR size 476: (TableCount (MatrixColsTable (MatrixFilterCols (MatrixMapRows (MatrixMapCols None (MatrixMapRows (MatrixMapCols None (MatrixAnnotateColsTable "__uid_3" (MatrixRead None False False "{\"name\":\"MatrixNativeReader\",\"path\":\"data/1kg.mt\",\"_spec\":null}") (TableKeyBy (Sample) False (TableRead None False "{\"name\":\"TextTableReader\",\"options\":{\"files\":[\"data/1kg_annotations.txt\"],\"typeMapStr\":{},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"impute\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"forceGZ\":false}}"))) (InsertFields (SelectFields (s) (Ref sa)) None (pheno (GetField __uid_3 (Ref sa))))) (InsertFields (SelectFields (locus alleles rsid qual filters info) (Ref va)) None (__uid_4 (GetField AC (ApplyAggOp CallStats ( (ArrayLen (GetField alleles (Ref va)))) ( (GetField GT (Ref g)))))) (__uid_5 (ArrayMap __uid_6 (ApplyIR `[*:]` Array[String] (GetField alleles (Ref va)) (I32 1)) (Let __uid_11 (ApplyIR __uid_1 Int32 (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (Ref __uid_6)) (If (ApplyComparisonOp EQ (Ref __uid_11) (I32 1)) (If (ApplySpecial `&&` Boolean (ApplyComparisonOp EQ (ApplyIR __uid_1 Int32 (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (Ref __uid_6)) (I32 1)) (ApplyIR __uid_2 Boolean (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (Ref __uid_6))) (I32 8) (I32 9)) (Ref __uid_11))))))) (AggLet __cse_3 False (GetField GT (Ref g)) (Let __cse_16 (ApplyIR toInt64 Int64 (I32 0)) (InsertFields (SelectFields (s pheno) (Ref sa)) None (sample_qc (Let __uid_27 (MakeStruct (n_called (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __cse_3))))))) (n_not_called (ApplyAggOp Sum () ( (Apply toInt64 Int64 (IsNA (Ref __cse_3)))))) (n_filtered (ApplyBinaryPrimOp Subtract (TableCount (MatrixRowsTable (MatrixMapRows (MatrixMapCols None (MatrixAnnotateColsTable "__uid_3" (MatrixRead None False False "{\"name\":\"MatrixNativeReader\",\"path\":\"data/1kg.mt\",\"_spec\":null}") (TableKeyBy (Sample) False (TableRead None False "{\"name\":\"TextTableReader\",\"options\":{\"files\":[\"data/1kg_annotations.txt\"],\"typeMapStr\":{},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"impute\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"forceGZ\":false}}"))) (InsertFields (SelectFields (s) (Ref sa)) None (pheno (GetField __uid_3 (Ref sa))))) (Let __cse_4 (GetField alleles (Ref va)) (InsertFields (SelectFields (locus alleles rsid qual filters info) (Ref va)) None (__uid_4 (GetField AC (ApplyAggOp CallStats ( (ArrayLen (GetField alleles (Ref va)))) ( (GetField GT (Ref g)))))) (__uid_5 (ArrayMap __uid_6 (ApplyIR `[*:]` Array[String] (Ref __cse_4) (I32 1)) (Let __uid_11 (ApplyIR __uid_1 Int32 (ApplyIR indexArray String (Ref __cse_4) (I32 0)) (Ref __uid_6)) (If (ApplyComparisonOp EQ (Ref __uid_11) (I32 1)) (Let __cse_5 (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (If (ApplySpecial `&&` Boolean (ApplyComparisonOp EQ (ApplyIR __uid_1 Int32 (Ref __cse_5) (Ref __uid_6)) (I32 1)) (ApplyIR __uid_2 Boolean (Ref __cse_5) (Ref __uid_6))) (I32 8) (I32 9))) (Ref __uid_11)))))))))) (ApplyAggOp Count () ()))) (n_hom_ref (ApplyAggOp Sum () ( (Apply toInt64 Int64 (Apply isHomRef Boolean (Ref __cse_3)))))) (n_het (ApplyAggOp Sum () ( (Apply toInt64 Int64 (Apply isHet Boolean (Ref __cse_3)))))) (n_singleton (ApplyAggOp Sum () ( (ApplyIR toInt64 Int64 (ApplyIR sum Int32 (ArrayFilter __uid_24 (ArrayMap __uid_23 (ArrayMap __uid_22 (ArrayRange (I32 0) (Apply ploidy Int32 (Ref __cse_3)) (I32 1)) (ApplyComparisonOp EQ (ApplyIR indexArray Int32 (GetField __uid_4 (Ref va)) (Apply `[]` Int32 (Ref __cse_3) (Ref __uid_22))) (I32 1))) (Apply toInt32 Int32 (Ref __uid_23))) (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_24))))))))) (allele_type_counts (AggExplode __uid_26 False (ArrayMap __uid_25 (ArrayRange (I32 0) (Apply ploidy Int32 (Ref __cse_3)) (I32 1)) (If (ApplyComparisonOp GT (Apply `[]` Int32 (Ref __cse_3) (Ref __uid_25)) (I32 0)) (ApplyIR indexArray Int32 (GetField __uid_5 (Ref va)) (ApplyBinaryPrimOp Subtract (Apply `[]` Int32 (GetField GT (Ref g)) (Ref __uid_25)) (I32 1))) (NA Int32))) (AggGroupBy False (Ref __uid_26) (ApplyAggOp Count () ()))))) (Let __cse_10 (GetField n_called (Ref __uid_27)) (Let __cse_11 (GetField n_not_called (Ref __uid_27)) (Let __cse_12 (GetField n_filtered (Ref __uid_27)) (Let __cse_13 (GetField n_hom_ref (Ref __uid_27)) (Let __cse_14 (GetField n_het (Ref __uid_27)) (Let __cse_15 (GetField allele_type_counts (Ref __uid_27)) (Let __uid_28 (MakeStruct (dp_stats (SelectFields (mean stdev min max) (AggLet __uid_agg16 False (ApplyIR toFloat64 Float64 (GetField DP (Ref g))) (Let __uid_17 (MakeStruct (n_def (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_agg16))))))) (sum (ApplyAggOp Sum () ( (Ref __uid_agg16)))) (sumsq (ApplyAggOp Sum () ( (Apply `**` Float64 (Ref __uid_agg16) (ApplyIR toFloat64 Float64 (I32 2)))))) (min (ApplyAggOp Min () ( (Ref __uid_agg16)))) (max (ApplyAggOp Max () ( (Ref __uid_agg16))))) (Let __cse_6 (GetField sum (Ref __uid_17)) (Let __cse_7 (GetField n_def (Ref __uid_17)) (Let __uid_18 (ApplyBinaryPrimOp FloatingPointDivide (Ref __cse_6) (ApplyIR toFloat64 Float64 (Ref __cse_7))) (MakeStruct (mean (Ref __uid_18)) (stdev (Apply sqrt Float64 (ApplyBinaryPrimOp FloatingPointDivide (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Subtract (GetField sumsq (Ref __uid_17)) (ApplyBinaryPrimOp Multiply (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (I32 2)) (Ref __uid_18)) (Ref __cse_6))) (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (Ref __cse_7)) (Apply `**` Float64 (Ref __uid_18) (ApplyIR toFloat64 Float64 (I32 2))))) (ApplyIR toFloat64 Float64 (Ref __cse_7))))) (min (GetField min (Ref __uid_17))) (max (GetField max (Ref __uid_17))) (n (Ref __cse_7)) (sum (Ref __cse_6)))))))))) (gq_stats (SelectFields (mean stdev min max) (AggLet __uid_agg19 False (ApplyIR toFloat64 Float64 (GetField GQ (Ref g))) (Let __uid_20 (MakeStruct (n_def (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_agg19))))))) (sum (ApplyAggOp Sum () ( (Ref __uid_agg19)))) (sumsq (ApplyAggOp Sum () ( (Apply `**` Float64 (Ref __uid_agg19) (ApplyIR toFloat64 Float64 (I32 2)))))) (min (ApplyAggOp Min () ( (Ref __uid_agg19)))) (max (ApplyAggOp Max () ( (Ref __uid_agg19))))) (Let __cse_8 (GetField sum (Ref __uid_20)) (Let __cse_9 (GetField n_def (Ref __uid_20)) (Let __uid_21 (ApplyBinaryPrimOp FloatingPointDivide (Ref __cse_8) (ApplyIR toFloat64 Float64 (Ref __cse_9))) (MakeStruct (mean (Ref __uid_21)) (stdev (Apply sqrt Float64 (ApplyBinaryPrimOp FloatingPointDivide (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Subtract (GetField sumsq (Ref __uid_20)) (ApplyBinaryPrimOp Multiply (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (I32 2)) (Ref __uid_21)) (Ref __cse_8))) (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (Ref __cse_9)) (Apply `**` Float64 (Ref __uid_21) (ApplyIR toFloat64 Float64 (I32 2))))) (ApplyIR toFloat64 Float64 (Ref __cse_9))))) (min (GetField min (Ref __uid_20))) (max (GetField max (Ref __uid_20))) (n (Ref __cse_9)) (sum (Ref __cse_8)))))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (Ref __cse_10)) (ApplyIR toFloat64 Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (Ref __cse_10) (Ref __cse_11)) (Ref __cse_12))))) (n_called (Ref __cse_10)) (n_not_called (Ref __cse_11)) (n_filtered (Ref __cse_12)) (n_hom_ref (Ref __cse_13)) (n_het (Ref __cse_14)) (n_hom_var (ApplyBinaryPrimOp Subtract (ApplyBinaryPrimOp Subtract (Ref __cse_10) (Ref __cse_13)) (Ref __cse_14))) (n_non_ref (ApplyBinaryPrimOp Subtract (Ref __cse_10) (Ref __cse_13))) (n_singleton (GetField n_singleton (Ref __uid_27))) (n_snp (ApplyBinaryPrimOp Add (ApplyIR get Int64 (Ref __cse_15) (I32 8) (Ref __cse_16)) (ApplyIR get Int64 (Ref __cse_15) (I32 9) (Ref __cse_16)))) (n_insertion (ApplyIR get Int64 (Ref __cse_15) (I32 3) (Ref __cse_16))) (n_deletion (ApplyIR get Int64 (Ref __cse_15) (I32 4) (Ref __cse_16))) (n_transition (ApplyIR get Int64 (Ref __cse_15) (I32 8) (Ref __cse_16))) (n_transversion (ApplyIR get Int64 (Ref __cse_15) (I32 9) (Ref __cse_16))) (n_star (ApplyIR get Int64 (Ref __cse_15) (I32 6) (Ref __cse_16)))) (InsertFields (Ref __uid_28) None (r_ti_tv (If (ApplyComparisonOp NEQ (GetField n_transversion (Ref __uid_28)) (ApplyIR toInt64 Int64 (I32 0))) (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (GetField n_transition (Ref __uid_28))) (ApplyIR toFloat64 Float64 (GetField n_transversion (Ref __uid_28)))) (NA Float64))) (r_het_hom_var (If (ApplyComparisonOp NEQ (GetField n_hom_var (Ref __uid_28)) (ApplyIR toInt64 Int64 (I32 0))) (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (GetField n_het (Ref __uid_28))) (ApplyIR toFloat64 Float64 (GetField n_hom_var (Ref __uid_28)))) (NA Float64))) (r_insertion_deletion (If (ApplyComparisonOp NEQ (GetField n_deletion (Ref __uid_28)) (ApplyIR toInt64 Int64 (I32 0))) (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (GetField n_insertion (Ref __uid_28))) (ApplyIR toFloat64 Float64 (GetField n_deletion (Ref __uid_28)))) (NA Float64))))))))))))))))) (SelectFields (locus alleles rsid qual filters info) (MakeStruct (locus (GetField locus (Ref va))) (alleles (GetField alleles (Ref va))) (rsid (GetField rsid (Ref va))) (qual (GetField qual (Ref va))) (filters (GetField filters (Ref va))) (info (GetField info (Ref va))) (__uid_4 (GetField __uid_4 (Ref va))) (__uid_5 (GetField __uid_5 (Ref va)))))) (Let __cse_17 (GetField sample_qc (Ref sa)) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (Ref __cse_17))) (ApplyIR toFloat64 Float64 (I32 4))) (ApplyComparisonOp GTEQ (GetField call_rate (Ref __cse_17)) (F64 0.97))) (False)))))) 2020-01-07 21:47:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 4.105ms, total 12.380ms 2020-01-07 21:47:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 19.222ms, total 31.782ms 2020-01-07 21:47:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 20.300ms, total 52.180ms 2020-01-07 21:47:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 52.319ms, total 104.610ms 2020-01-07 21:47:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 3.377ms, total 108.127ms 2020-01-07 21:47:37 root: INFO: Prune: InsertFields: eliminating field 'pheno' 2020-01-07 21:47:37 root: INFO: Prune: InsertFields: eliminating field '__uid_4' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'n_singleton' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'sumsq' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'min' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'max' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'stdev' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'min' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'max' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'n' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'sum' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'gq_stats' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'n_called' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'n_not_called' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'n_filtered' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'n_hom_ref' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'n_het' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'n_hom_var' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'n_non_ref' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'n_singleton' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'n_snp' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'n_insertion' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'n_deletion' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'n_transition' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'n_transversion' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'n_star' 2020-01-07 21:47:37 root: INFO: Prune: InsertFields: eliminating field 'r_ti_tv' 2020-01-07 21:47:37 root: INFO: Prune: InsertFields: eliminating field 'r_het_hom_var' 2020-01-07 21:47:37 root: INFO: Prune: InsertFields: eliminating field 'r_insertion_deletion' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'locus' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'alleles' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'rsid' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'qual' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'filters' 2020-01-07 21:47:37 root: INFO: Prune: MakeStruct: eliminating field 'info' 2020-01-07 21:47:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 46.858ms, total 155.084ms 2020-01-07 21:47:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 1.648ms, total 156.892ms 2020-01-07 21:47:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 9.291ms, total 166.288ms 2020-01-07 21:47:37 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 3.561ms, total 169.984ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 32.883ms, total 202.978ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 1.434ms, total 204.558ms 2020-01-07 21:47:38 root: INFO: Prune: InsertFields: eliminating field '__uid_5' 2020-01-07 21:47:38 root: INFO: Prune: MakeStruct: eliminating field 'n_hom_ref' 2020-01-07 21:47:38 root: INFO: Prune: MakeStruct: eliminating field 'n_het' 2020-01-07 21:47:38 root: INFO: Prune: MakeStruct: eliminating field 'allele_type_counts' 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 5.552ms, total 210.193ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.342ms, total 210.664ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.337ms, total 211.059ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 3.981ms, total 215.100ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 3.187ms, total 218.355ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 1.305ms, total 219.730ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 3.588ms, total 223.374ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 215.168ms, total 223.436ms, tagged coverage 99.1 2020-01-07 21:47:38 root: INFO: optimize Lowerer, initial IR: after: IR size 80: (TableCount (MatrixColsTable (MatrixFilterCols (MatrixMapRows (MatrixKeyRowsBy () False (MatrixMapCols () (MatrixRead Matrix{global:Struct{},col_key:[s],col:Struct{s:String},row_key:[[locus,alleles]],row:Struct{locus:Locus(GRCh37),alleles:Array[String]},entry:Struct{GT:Call,DP:Int32}} False False "{\"name\":\"MatrixNativeReader\",\"path\":\"data/1kg.mt\",\"_spec\":null}") (AggLet __iruid_1432 False (GetField GT (Ref g)) (InsertFields (SelectFields () (Ref sa)) None (sample_qc (Let __iruid_1433 (MakeStruct (n_called (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_1432))))))) (n_not_called (ApplyAggOp Sum () ( (Apply toInt64 Int64 (IsNA (Ref __iruid_1432)))))) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (ApplyAggOp Count () ())))) (Let __iruid_1434 (GetField n_called (Ref __iruid_1433)) (Let __iruid_1435 (MakeStruct (dp_stats (AggLet __iruid_1436 False (Cast Float64 (GetField DP (Ref g))) (Let __iruid_1437 (MakeStruct (n_def (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_1436))))))) (sum (ApplyAggOp Sum () ( (Ref __iruid_1436))))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_1437)) (Cast Float64 (GetField n_def (Ref __iruid_1437))))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (Ref __iruid_1434)) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (Ref __iruid_1434) (GetField n_not_called (Ref __iruid_1433))) (GetField n_filtered (Ref __iruid_1433))))))) (Ref __iruid_1435))))))))) (Literal Struct{} )) (Let __iruid_1438 (GetField sample_qc (Ref sa)) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (Ref __iruid_1438))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (Ref __iruid_1438)) (F64 0.97))) (False)))))) 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- Verify : 0.095ms, total 225.135ms 2020-01-07 21:47:38 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:38 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:38 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- LoweringTransformation : 35.910ms, total 261.125ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- Verify : 1.217ms, total 262.442ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable : 37.474ms, total 262.511ms, tagged coverage 99.3 2020-01-07 21:47:38 root: INFO: optimize LowerMatrixToTable, post Lowering: before: IR size 228: (RelationalLet __iruid_1447 (TableAggregate (TableMapGlobals (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,DP:Int32}]}} False "{\"name\":\"TableNativeZippedReader\",\"pathLeft\":\"data/1kg.mt/rows\",\"pathRight\":\"data/1kg.mt/entries\",\"_specLeft\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AC:Array[Int32],AF:Array[Float64],AN:Int32,BaseQRankSum:Float64,ClippingRankSum:Float64,DP:Int32,DS:Boolean,FS:Float64,HaplotypeScore:Float64,InbreedingCoeff:Float64,MLEAC:Array[Int32],MLEAF:Array[Float64],MQ:Float64,MQ0:Int32,MQRankSum:Float64,QD:Float64,ReadPosRankSum:Float64,set:String}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}},\"_specRight\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[],row:Struct{`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32,GQ:Int32,PL:Array[+Int32]}]}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}}}") (InsertFields (Ref global) None (__cols (GetField rows (TableCollect (TableRead Table{global:Struct{},key:[],row:Struct{s:String}} False "{\"name\":\"TableNativeReader\",\"path\":\"data/1kg.mt/cols\",\"_spec\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[s],row:Struct{s:String}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[284]}}}}")))))) (AggLet va False (SelectFields (locus alleles) (Ref row)) (AggArrayPerElement __element_idx __result_idx False True (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let sa (ArrayRef (GetField __cols (Ref global)) (Ref __result_idx)) (AggLet sa False (ArrayRef (GetField __cols (Ref global)) (Ref __element_idx)) (AggLet g False (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __element_idx)) (AggFilter False (ApplyUnaryPrimOp Bang (IsNA (Ref g))) (MakeStruct (__iruid_1445 (AggLet __iruid_1432 False (GetField GT (Ref g)) (MakeStruct (__iruid_1439 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_1432))))))) (__iruid_1440 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (IsNA (Ref __iruid_1432)))))) (__iruid_1441 (ApplyAggOp Count () ())) (__iruid_1444 (AggLet __iruid_1436 False (Cast Float64 (GetField DP (Ref g))) (MakeStruct (__iruid_1442 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_1436))))))) (__iruid_1443 (ApplyAggOp Sum () ( (Ref __iruid_1436))))))))))))))) (ArrayLen (GetField __cols (Ref global)))))) (TableCount (TableKeyBy () False (TableParallelize None (Let __cols_and_globals (TableGetGlobals (TableMapGlobals (TableMapRows (TableMapGlobals (TableMapRows (TableKeyBy () False (TableMapGlobals (TableMapGlobals (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,DP:Int32}]}} False "{\"name\":\"TableNativeZippedReader\",\"pathLeft\":\"data/1kg.mt/rows\",\"pathRight\":\"data/1kg.mt/entries\",\"_specLeft\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AC:Array[Int32],AF:Array[Float64],AN:Int32,BaseQRankSum:Float64,ClippingRankSum:Float64,DP:Int32,DS:Boolean,FS:Float64,HaplotypeScore:Float64,InbreedingCoeff:Float64,MLEAC:Array[Int32],MLEAF:Array[Float64],MQ:Float64,MQ0:Int32,MQRankSum:Float64,QD:Float64,ReadPosRankSum:Float64,set:String}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}},\"_specRight\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[],row:Struct{`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32,GQ:Int32,PL:Array[+Int32]}]}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}}}") (InsertFields (Ref global) None (__cols (GetField rows (TableCollect (TableRead Table{global:Struct{},key:[],row:Struct{s:String}} False "{\"name\":\"TableNativeReader\",\"path\":\"data/1kg.mt/cols\",\"_spec\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[s],row:Struct{s:String}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[284]}}}}")))))) (InsertFields (Ref global) None (__cols (Let __iruid_1448 (RelationalRef __iruid_1447 Array[Struct{__iruid_1445:Struct{__iruid_1439:Int64,__iruid_1440:Int64,__iruid_1441:Int64,__iruid_1444:Struct{__iruid_1442:Int64,__iruid_1443:Float64}}}]) (ArrayMap __iruid_1446 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __cols_array (GetField __cols (Ref global)) (Let sa (ArrayRef (Ref __cols_array) (Ref __iruid_1446)) (Let __iruid_1448 (RelationalRef __iruid_1447 Array[Struct{__iruid_1445:Struct{__iruid_1439:Int64,__iruid_1440:Int64,__iruid_1441:Int64,__iruid_1444:Struct{__iruid_1442:Int64,__iruid_1443:Float64}}}]) (Let __iruid_1449 (ArrayRef (Ref __iruid_1448) (Ref __iruid_1446)) (Let __iruid_1445 (GetField __iruid_1445 (Ref __iruid_1449)) (Let __iruid_1444 (GetField __iruid_1444 (Ref __iruid_1445)) (Let __iruid_1441 (GetField __iruid_1441 (Ref __iruid_1445)) (Let __iruid_1440 (GetField __iruid_1440 (Ref __iruid_1445)) (Let __iruid_1439 (GetField __iruid_1439 (Ref __iruid_1445)) (InsertFields (SelectFields () (Ref sa)) None (sample_qc (Let __iruid_1433 (MakeStruct (n_called (Ref __iruid_1439)) (n_not_called (Ref __iruid_1440)) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (Ref __iruid_1441)))) (Let __iruid_1434 (GetField n_called (Ref __iruid_1433)) (Let __iruid_1435 (MakeStruct (dp_stats (Let __iruid_1443 (GetField __iruid_1443 (Ref __iruid_1444)) (Let __iruid_1442 (GetField __iruid_1442 (Ref __iruid_1444)) (Let __iruid_1437 (MakeStruct (n_def (Ref __iruid_1442)) (sum (Ref __iruid_1443))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_1437)) (Cast Float64 (GetField n_def (Ref __iruid_1437)))))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (Ref __iruid_1434)) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (Ref __iruid_1434) (GetField n_not_called (Ref __iruid_1433))) (GetField n_filtered (Ref __iruid_1433))))))) (Ref __iruid_1435))))))))))))))))))))) (InsertFields (Let __iruid_1450 (MakeStruct) (Literal Struct{} )) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row))))) (InsertFields (Ref global) None (newColIdx (ArrayFilter i (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let sa (ArrayRef (GetField __cols (Ref global)) (Ref i)) (Let __iruid_1438 (GetField sample_qc (Ref sa)) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (Ref __iruid_1438))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (Ref __iruid_1438)) (F64 0.97))) (False)))))))) (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap i (GetField newColIdx (Ref global)) (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref i)))))) (SelectFields (__cols) (InsertFields (Ref global) None (__cols (ArrayMap i (GetField newColIdx (Ref global)) (ArrayRef (GetField __cols (Ref global)) (Ref i)))))))) (MakeStruct (rows (GetField __cols (Ref __cols_and_globals))) (global (SelectFields () (Ref __cols_and_globals))))))))) 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 1.292ms, total 273.853ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 1.220ms, total 275.133ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 34.845ms, total 310.031ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 8.893ms, total 319.010ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 1.461ms, total 320.544ms 2020-01-07 21:47:38 root: INFO: Prune: MakeStruct: eliminating field 'global' 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 18.440ms, total 339.042ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.490ms, total 339.606ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.409ms, total 340.081ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.985ms, total 341.115ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 5.667ms, total 346.833ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.475ms, total 347.361ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 4.885ms, total 352.310ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.692ms, total 353.194ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.341ms, total 353.599ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.914ms, total 354.581ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 5.549ms, total 360.213ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.531ms, total 360.820ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 4.443ms, total 365.315ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 92.816ms, total 365.371ms, tagged coverage 98.6 2020-01-07 21:47:38 root: INFO: optimize LowerMatrixToTable, post Lowering: after: IR size 148: (RelationalLet __iruid_1447 (TableAggregate (TableKeyBy () False (TableMapGlobals (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,DP:Int32}]}} False "{\"name\":\"TableNativeZippedReader\",\"pathLeft\":\"data/1kg.mt/rows\",\"pathRight\":\"data/1kg.mt/entries\",\"_specLeft\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AC:Array[Int32],AF:Array[Float64],AN:Int32,BaseQRankSum:Float64,ClippingRankSum:Float64,DP:Int32,DS:Boolean,FS:Float64,HaplotypeScore:Float64,InbreedingCoeff:Float64,MLEAC:Array[Int32],MLEAF:Array[Float64],MQ:Float64,MQ0:Int32,MQRankSum:Float64,QD:Float64,ReadPosRankSum:Float64,set:String}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}},\"_specRight\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[],row:Struct{`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32,GQ:Int32,PL:Array[+Int32]}]}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}}}") (InsertFields (Ref global) None (__cols (GetField rows (TableCollect (TableRead Table{global:Struct{},key:[],row:Struct{}} False "{\"name\":\"TableNativeReader\",\"path\":\"data/1kg.mt/cols\",\"_spec\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[s],row:Struct{s:String}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[284]}}}}"))))))) (AggArrayPerElement __iruid_1537 __iruid_1538 False True (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (AggLet __iruid_1539 False (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_1537)) (AggFilter False (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_1539))) (MakeStruct (__iruid_1445 (AggLet __iruid_1540 False (GetField GT (Ref __iruid_1539)) (MakeStruct (__iruid_1439 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_1540))))))) (__iruid_1440 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (IsNA (Ref __iruid_1540)))))) (__iruid_1441 (ApplyAggOp Count () ())) (__iruid_1444 (AggLet __iruid_1541 False (Cast Float64 (GetField DP (Ref __iruid_1539))) (MakeStruct (__iruid_1442 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_1541))))))) (__iruid_1443 (ApplyAggOp Sum () ( (Ref __iruid_1541))))))))))))) (ArrayLen (GetField __cols (Ref global))))) (Cast Int64 (ArrayLen (GetField rows (Let __iruid_1542 (GetField rows (TableCollect (TableRead Table{global:Struct{},key:[],row:Struct{}} False "{\"name\":\"TableNativeReader\",\"path\":\"data/1kg.mt/cols\",\"_spec\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[s],row:Struct{s:String}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[284]}}}}"))) (Let __iruid_1543 (ArrayMap __iruid_1544 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_1542)) (I32 1)) (Let __iruid_1545 (GetField __iruid_1445 (ArrayRef (RelationalRef __iruid_1447 Array[Struct{__iruid_1445:Struct{__iruid_1439:Int64,__iruid_1440:Int64,__iruid_1441:Int64,__iruid_1444:Struct{__iruid_1442:Int64,__iruid_1443:Float64}}}]) (Ref __iruid_1544))) (Let __iruid_1546 (GetField __iruid_1444 (Ref __iruid_1545)) (InsertFields (ArrayRef (Ref __iruid_1542) (Ref __iruid_1544)) None (sample_qc (Let __iruid_1547 (MakeStruct (n_called (GetField __iruid_1439 (Ref __iruid_1545))) (n_not_called (GetField __iruid_1440 (Ref __iruid_1545))) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (GetField __iruid_1441 (Ref __iruid_1545))))) (Let __iruid_1548 (GetField n_called (Ref __iruid_1547)) (MakeStruct (dp_stats (Let __iruid_1549 (MakeStruct (n_def (GetField __iruid_1442 (Ref __iruid_1546))) (sum (GetField __iruid_1443 (Ref __iruid_1546)))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_1549)) (Cast Float64 (GetField n_def (Ref __iruid_1549)))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (Ref __iruid_1548)) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (Ref __iruid_1548) (GetField n_not_called (Ref __iruid_1547))) (GetField n_filtered (Ref __iruid_1547)))))))))))))) (MakeStruct (rows (ArrayMap __iruid_1550 (ArrayFilter __iruid_1551 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_1543)) (I32 1)) (Let __iruid_1552 (GetField sample_qc (ArrayRef (Ref __iruid_1543) (Ref __iruid_1551))) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (Ref __iruid_1552))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (Ref __iruid_1552)) (F64 0.97))) (False)))) (ArrayRef (Ref __iruid_1543) (Ref __iruid_1550))))))))))) 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- Verify : 0.119ms, total 368.869ms 2020-01-07 21:47:38 root: INFO: interpreting non compilable node: TableCollect 2020-01-07 21:47:38 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:38 root: INFO: decoder cache hit 2020-01-07 21:47:38 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:38 root: INFO: decoder cache miss (2 hits, 3 misses, 0.400 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeDecode_17. 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeDecode_17.apply 2020-01-07 21:47:38 root: INFO: instruction count: 27: is/hail/codegen/generated/C_etypeDecode_17.m0_DECODE_o_struct_of_o_binaryEND_TO_o_tuple_of_END 2020-01-07 21:47:38 root: INFO: instruction count: 17: is/hail/codegen/generated/C_etypeDecode_17.m1_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 9: is/hail/codegen/generated/C_etypeDecode_17.m2_SKIP_o_binary 2020-01-07 21:47:38 root: INFO: encoder cache hit 2020-01-07 21:47:38 SparkContext: INFO: Starting job: collect at RVD.scala:676 2020-01-07 21:47:38 DAGScheduler: INFO: Got job 5 (collect at RVD.scala:676) with 1 output partitions 2020-01-07 21:47:38 DAGScheduler: INFO: Final stage: ResultStage 5 (collect at RVD.scala:676) 2020-01-07 21:47:38 DAGScheduler: INFO: Parents of final stage: List() 2020-01-07 21:47:38 DAGScheduler: INFO: Missing parents: List() 2020-01-07 21:47:38 DAGScheduler: INFO: Submitting ResultStage 5 (MapPartitionsRDD[27] at mapPartitions at ContextRDD.scala:217), which has no missing parents 2020-01-07 21:47:38 MemoryStore: INFO: Block broadcast_12 stored as values in memory (estimated size 10.3 KB, free 365.7 MB) 2020-01-07 21:47:38 MemoryStore: INFO: Block broadcast_12_piece0 stored as bytes in memory (estimated size 5.5 KB, free 365.7 MB) 2020-01-07 21:47:38 BlockManagerInfo: INFO: Added broadcast_12_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 5.5 KB, free: 366.2 MB) 2020-01-07 21:47:38 SparkContext: INFO: Created broadcast 12 from broadcast at DAGScheduler.scala:1161 2020-01-07 21:47:38 DAGScheduler: INFO: Submitting 1 missing tasks from ResultStage 5 (MapPartitionsRDD[27] at mapPartitions at ContextRDD.scala:217) (first 15 tasks are for partitions Vector(0)) 2020-01-07 21:47:38 TaskSchedulerImpl: INFO: Adding task set 5.0 with 1 tasks 2020-01-07 21:47:38 TaskSetManager: INFO: Starting task 0.0 in stage 5.0 (TID 39, localhost, executor driver, partition 0, PROCESS_LOCAL, 7653 bytes) 2020-01-07 21:47:38 Executor: INFO: Running task 0.0 in stage 5.0 (TID 39) 2020-01-07 21:47:38 root: INFO: RegionPool: initialized for thread 225: Executor task launch worker for task 39 2020-01-07 21:47:38 Executor: INFO: Finished task 0.0 in stage 5.0 (TID 39). 3940 bytes result sent to driver 2020-01-07 21:47:38 TaskSetManager: INFO: Finished task 0.0 in stage 5.0 (TID 39) in 18 ms on localhost (executor driver) (1/1) 2020-01-07 21:47:38 TaskSchedulerImpl: INFO: Removed TaskSet 5.0, whose tasks have all completed, from pool 2020-01-07 21:47:38 DAGScheduler: INFO: ResultStage 5 (collect at RVD.scala:676) finished in 0.032 s 2020-01-07 21:47:38 DAGScheduler: INFO: Job 5 finished: collect at RVD.scala:676, took 0.036311 s 2020-01-07 21:47:38 root: INFO: decoder cache hit 2020-01-07 21:47:38 root: INFO: took 72.325ms 2020-01-07 21:47:38 root: INFO: interpreting non compilable node: TableAggregate 2020-01-07 21:47:38 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:38 root: INFO: decoder cache hit 2020-01-07 21:47:38 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:38 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:38 root: INFO: decoder cache hit 2020-01-07 21:47:38 root: INFO: decoder cache miss (5 hits, 4 misses, 0.556 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeDecode_18. 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeDecode_18.apply 2020-01-07 21:47:38 root: INFO: instruction count: 27: is/hail/codegen/generated/C_etypeDecode_18.m0_DECODE_o_struct_of_o_array_of_o_struct_of_o_int32ANDo_array_of_r_int32ANDo_int32ANDo_int32ANDo_array_of_r_int32ENDEND_TO_o_tuple_of_o_array_of_o_tuple_of_o_callANDo_int32ENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_18.m1_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 97: is/hail/codegen/generated/C_etypeDecode_18.m2_INPLACE_DECODE_o_array_of_o_struct_of_o_int32ANDo_array_of_r_int32ANDo_int32ANDo_int32ANDo_array_of_r_int32END_TO_o_array_of_o_tuple_of_o_int32ANDo_int32END 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_18.m3_INPLACE_DECODE_o_struct_of_o_int32ANDo_array_of_r_int32ANDo_int32ANDo_int32ANDo_array_of_r_int32END_TO_o_tuple_of_o_int32ANDo_int32END 2020-01-07 21:47:38 root: INFO: instruction count: 95: is/hail/codegen/generated/C_etypeDecode_18.m4_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_18.m5_INPLACE_DECODE_o_int32_TO_o_int32 2020-01-07 21:47:38 root: INFO: instruction count: 25: is/hail/codegen/generated/C_etypeDecode_18.m6_SKIP_o_array_of_r_int32 2020-01-07 21:47:38 root: INFO: instruction count: 5: is/hail/codegen/generated/C_etypeDecode_18.m7_SKIP_r_int32 2020-01-07 21:47:38 root: INFO: instruction count: 5: is/hail/codegen/generated/C_etypeDecode_18.m8_SKIP_o_int32 2020-01-07 21:47:38 root: INFO: optimize Compile: before: IR size 4: (InsertFields (Ref left) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref right)))) 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.103ms, total 475.395ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.103ms, total 475.584ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.069ms, total 475.725ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.317ms, total 476.090ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.044ms, total 476.197ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.541ms, total 476.786ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 1.589ms, total 476.876ms, tagged coverage 74.0 2020-01-07 21:47:38 root: INFO: optimize Compile: after: IR size 4: (InsertFields (Ref left) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref right)))) 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C19. 2020-01-07 21:47:38 root: INFO: instruction count: 142: is/hail/codegen/generated/C19.apply 2020-01-07 21:47:38 root: INFO: instruction count: 22: is/hail/codegen/generated/C19.apply 2020-01-07 21:47:38 root: INFO: instruction count: 14: is/hail/codegen/generated/C19.setPartitionIndex 2020-01-07 21:47:38 root: INFO: instruction count: 6: is/hail/codegen/generated/C19.addPartitionRegion 2020-01-07 21:47:38 root: INFO: decoder cache miss (5 hits, 5 misses, 0.500 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeDecode_20. 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeDecode_20.apply 2020-01-07 21:47:38 root: INFO: instruction count: 20: is/hail/codegen/generated/C_etypeDecode_20.m0_DECODE_o_struct_of_r_int64ANDr_array_of_r_struct_of_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDENDEND_TO_o_tuple_of_r_int64ANDr_array_of_r_tuple_of_o_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_o_int64ENDENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 17: is/hail/codegen/generated/C_etypeDecode_20.m1_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_20.m2_INPLACE_DECODE_r_int64_TO_r_int64 2020-01-07 21:47:38 root: INFO: instruction count: 51: is/hail/codegen/generated/C_etypeDecode_20.m3_INPLACE_DECODE_r_array_of_r_struct_of_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDEND_TO_r_array_of_r_tuple_of_o_tuple_of_o_tuple_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_tuple_of_o_int64ENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_20.m4_INPLACE_DECODE_r_struct_of_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDEND_TO_r_tuple_of_o_tuple_of_o_tuple_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_tuple_of_o_int64ENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 42: is/hail/codegen/generated/C_etypeDecode_20.m5_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_20.m6_INPLACE_DECODE_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryEND_TO_o_tuple_of_o_tuple_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryEND 2020-01-07 21:47:38 root: INFO: instruction count: 53: is/hail/codegen/generated/C_etypeDecode_20.m7_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 14: is/hail/codegen/generated/C_etypeDecode_20.m8_INPLACE_DECODE_o_struct_of_r_binaryANDr_int32END_TO_o_tuple_of_r_binaryANDr_int32END 2020-01-07 21:47:38 root: INFO: instruction count: 17: is/hail/codegen/generated/C_etypeDecode_20.m9_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 29: is/hail/codegen/generated/C_etypeDecode_20.m10_INPLACE_DECODE_r_binary_TO_r_binary 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_20.m11_INPLACE_DECODE_r_int32_TO_r_int32 2020-01-07 21:47:38 root: INFO: instruction count: 97: is/hail/codegen/generated/C_etypeDecode_20.m12_INPLACE_DECODE_o_array_of_o_binary_TO_o_array_of_o_binary 2020-01-07 21:47:38 root: INFO: instruction count: 29: is/hail/codegen/generated/C_etypeDecode_20.m13_INPLACE_DECODE_o_binary_TO_o_binary 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_20.m14_INPLACE_DECODE_r_struct_of_o_int64END_TO_r_tuple_of_o_int64END 2020-01-07 21:47:38 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_20.m15_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_20.m16_INPLACE_DECODE_o_int64_TO_o_int64 2020-01-07 21:47:38 root: INFO: decoder cache miss (5 hits, 6 misses, 0.455 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeDecode_21. 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeDecode_21.apply 2020-01-07 21:47:38 root: INFO: instruction count: 20: is/hail/codegen/generated/C_etypeDecode_21.m0_DECODE_o_struct_of_r_array_of_r_struct_of_r_int64ANDr_int64ANDo_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDENDEND_TO_o_tuple_of_r_array_of_r_tuple_of_r_int64ANDr_int64ANDo_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_o_int64ENDENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 10: is/hail/codegen/generated/C_etypeDecode_21.m1_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 51: is/hail/codegen/generated/C_etypeDecode_21.m2_INPLACE_DECODE_r_array_of_r_struct_of_r_int64ANDr_int64ANDo_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDEND_TO_r_array_of_r_tuple_of_r_int64ANDr_int64ANDo_tuple_of_o_tuple_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_tuple_of_o_int64ENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_21.m3_INPLACE_DECODE_r_struct_of_r_int64ANDr_int64ANDo_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDEND_TO_r_tuple_of_r_int64ANDr_int64ANDo_tuple_of_o_tuple_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_tuple_of_o_int64ENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 56: is/hail/codegen/generated/C_etypeDecode_21.m4_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_21.m5_INPLACE_DECODE_r_int64_TO_r_int64 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_21.m6_INPLACE_DECODE_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryEND_TO_o_tuple_of_o_tuple_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryEND 2020-01-07 21:47:38 root: INFO: instruction count: 53: is/hail/codegen/generated/C_etypeDecode_21.m7_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 14: is/hail/codegen/generated/C_etypeDecode_21.m8_INPLACE_DECODE_o_struct_of_r_binaryANDr_int32END_TO_o_tuple_of_r_binaryANDr_int32END 2020-01-07 21:47:38 root: INFO: instruction count: 17: is/hail/codegen/generated/C_etypeDecode_21.m9_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 29: is/hail/codegen/generated/C_etypeDecode_21.m10_INPLACE_DECODE_r_binary_TO_r_binary 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_21.m11_INPLACE_DECODE_r_int32_TO_r_int32 2020-01-07 21:47:38 root: INFO: instruction count: 97: is/hail/codegen/generated/C_etypeDecode_21.m12_INPLACE_DECODE_o_array_of_o_binary_TO_o_array_of_o_binary 2020-01-07 21:47:38 root: INFO: instruction count: 29: is/hail/codegen/generated/C_etypeDecode_21.m13_INPLACE_DECODE_o_binary_TO_o_binary 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_21.m14_INPLACE_DECODE_r_struct_of_o_int64END_TO_r_tuple_of_o_int64END 2020-01-07 21:47:38 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_21.m15_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_21.m16_INPLACE_DECODE_o_int64_TO_o_int64 2020-01-07 21:47:38 root: INFO: optimize Compile: before: IR size 4: (InsertFields (Ref global) None (__cols (GetField rows (Literal Struct{rows:Array[Struct{}],global:Struct{}} )))) 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.208ms, total 506.713ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.030ms, total 506.819ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.044ms, total 506.907ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.253ms, total 507.223ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.046ms, total 507.325ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.179ms, total 507.549ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.112ms, total 507.717ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.041ms, total 507.840ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.031ms, total 507.909ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.152ms, total 508.096ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.037ms, total 508.179ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.144ms, total 508.377ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 1.926ms, total 508.425ms, tagged coverage 66.4 2020-01-07 21:47:38 root: INFO: optimize Compile: after: IR size 3: (InsertFields (Ref global) None (__cols (Literal Array[Struct{}] ))) 2020-01-07 21:47:38 root: INFO: encoder cache miss (3 hits, 8 misses, 0.273) 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_23. 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_23.apply 2020-01-07 21:47:38 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_23.m0_ENCODE_o_tuple_of_o_array_of_o_tuple_of_ENDEND_TO_o_struct_of_o_array_of_o_struct_of_ENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_23.m1_write_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 74: is/hail/codegen/generated/C_etypeEncode_23.m2_ENCODE_o_array_of_o_tuple_of_END_TO_o_array_of_o_struct_of_END 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_23.m3_ENCODE_o_tuple_of_END_TO_o_struct_of_END 2020-01-07 21:47:38 MemoryStore: INFO: Block broadcast_13 stored as values in memory (estimated size 96.0 B, free 365.7 MB) 2020-01-07 21:47:38 MemoryStore: INFO: Block broadcast_13_piece0 stored as bytes in memory (estimated size 65.0 B, free 365.7 MB) 2020-01-07 21:47:38 BlockManagerInfo: INFO: Added broadcast_13_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 65.0 B, free: 366.2 MB) 2020-01-07 21:47:38 SparkContext: INFO: Created broadcast 13 from broadcast at SparkBackend.scala:21 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C22. 2020-01-07 21:47:38 root: INFO: instruction count: 57: is/hail/codegen/generated/C22.apply 2020-01-07 21:47:38 root: INFO: instruction count: 16: is/hail/codegen/generated/C22.apply 2020-01-07 21:47:38 root: INFO: instruction count: 14: is/hail/codegen/generated/C22.setPartitionIndex 2020-01-07 21:47:38 root: INFO: instruction count: 6: is/hail/codegen/generated/C22.addPartitionRegion 2020-01-07 21:47:38 root: INFO: instruction count: 27: is/hail/codegen/generated/C22.m3_DECODE_o_struct_of_o_array_of_o_struct_of_ENDEND_TO_o_tuple_of_o_array_of_o_tuple_of_ENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 28: is/hail/codegen/generated/C22.m4_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 97: is/hail/codegen/generated/C22.m5_INPLACE_DECODE_o_array_of_o_struct_of_END_TO_o_array_of_o_tuple_of_END 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C22.m6_INPLACE_DECODE_o_struct_of_END_TO_o_tuple_of_END 2020-01-07 21:47:38 root: INFO: instruction count: 30: is/hail/codegen/generated/C22.addLiterals 2020-01-07 21:47:38 root: INFO: encoder cache miss (3 hits, 9 misses, 0.250) 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_24. 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_24.apply 2020-01-07 21:47:38 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_24.m0_ENCODE_o_tuple_of_o_array_of_o_tuple_of_ENDEND_TO_o_struct_of_o_array_of_o_struct_of_ENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_24.m1_write_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 74: is/hail/codegen/generated/C_etypeEncode_24.m2_ENCODE_o_array_of_o_tuple_of_END_TO_o_array_of_o_struct_of_END 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_24.m3_ENCODE_o_tuple_of_END_TO_o_struct_of_END 2020-01-07 21:47:38 root: INFO: decoder cache miss (5 hits, 7 misses, 0.417 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeDecode_25. 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeDecode_25.apply 2020-01-07 21:47:38 root: INFO: instruction count: 27: is/hail/codegen/generated/C_etypeDecode_25.m0_DECODE_o_struct_of_o_array_of_o_struct_of_ENDEND_TO_o_tuple_of_o_array_of_o_tuple_of_ENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_25.m1_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 97: is/hail/codegen/generated/C_etypeDecode_25.m2_INPLACE_DECODE_o_array_of_o_struct_of_END_TO_o_array_of_o_tuple_of_END 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_25.m3_INPLACE_DECODE_o_struct_of_END_TO_o_tuple_of_END 2020-01-07 21:47:38 MemoryStore: INFO: Block broadcast_14 stored as values in memory (estimated size 4.4 KB, free 365.7 MB) 2020-01-07 21:47:38 MemoryStore: INFO: Block broadcast_14_piece0 stored as bytes in memory (estimated size 2.1 KB, free 365.7 MB) 2020-01-07 21:47:38 BlockManagerInfo: INFO: Added broadcast_14_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 2.1 KB, free: 366.2 MB) 2020-01-07 21:47:38 SparkContext: INFO: Created broadcast 14 from broadcast at SparkBackend.scala:21 2020-01-07 21:47:38 root: INFO: optimize Compile: before: IR size 11: (Begin (InitOp2 0 (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))) ( (ArrayLen (GetField __cols (Ref global))) (Begin (InitOp2 0 (Sum () (Int64) None) ()) (InitOp2 1 (Sum () (Int64) None) ()) (InitOp2 2 (Count () () None) ()) (InitOp2 3 (Sum () (Int64) None) ()) (InitOp2 4 (Sum () (Float64) None) ()))))) 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.088ms, total 533.108ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.055ms, total 533.227ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.080ms, total 533.351ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.395ms, total 533.785ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.048ms, total 533.881ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.224ms, total 534.145ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 1.195ms, total 534.209ms, tagged coverage 74.5 2020-01-07 21:47:38 root: INFO: optimize Compile: after: IR size 11: (Begin (InitOp2 0 (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))) ( (ArrayLen (GetField __cols (Ref global))) (Begin (InitOp2 0 (Sum () (Int64) None) ()) (InitOp2 1 (Sum () (Int64) None) ()) (InitOp2 2 (Count () () None) ()) (InitOp2 3 (Sum () (Int64) None) ()) (InitOp2 4 (Sum () (Float64) None) ()))))) 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C26. 2020-01-07 21:47:38 root: INFO: instruction count: 20: is/hail/codegen/generated/C26.apply 2020-01-07 21:47:38 root: INFO: instruction count: 14: is/hail/codegen/generated/C26.apply 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C26.newAggState 2020-01-07 21:47:38 root: INFO: instruction count: 17: is/hail/codegen/generated/C26.setAggState 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C26.getAggOffset 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C26.setNumSerialized 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C26.setSerializedAgg 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C26.getSerializedAgg 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C26.m7_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 19: is/hail/codegen/generated/C26.m8_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C26.m9_new_state_group0 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C26.m10_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 19: is/hail/codegen/generated/C26.m11_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 53: is/hail/codegen/generated/C26.m12_load_group0 2020-01-07 21:47:38 root: INFO: instruction count: 36: is/hail/codegen/generated/C26.m13_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 140: is/hail/codegen/generated/C26.m14_method 2020-01-07 21:47:38 root: INFO: instruction count: 18: is/hail/codegen/generated/C26.m15_method 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C26.m16_new_state_group0 2020-01-07 21:47:38 root: INFO: instruction count: 63: is/hail/codegen/generated/C26.m17_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 48: is/hail/codegen/generated/C26.m18_copy_from_group0 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C26.m19_new_state_group0 2020-01-07 21:47:38 root: INFO: instruction count: 113: is/hail/codegen/generated/C26.m20_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 14: is/hail/codegen/generated/C26.setPartitionIndex 2020-01-07 21:47:38 root: INFO: instruction count: 6: is/hail/codegen/generated/C26.addPartitionRegion 2020-01-07 21:47:38 root: INFO: optimize Compile: before: IR size 60: (Begin (Let __iruid_1555 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Begin (SeqOp2 0 (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))) ( (ArrayLen (Ref __iruid_1555)))) (ArrayFor __iruid_1556 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_1555)) (I32 1)) (Let __iruid_1537 (ArrayRef (Ref __iruid_1555) (Ref __iruid_1556)) (Let __iruid_1539 (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_1537)) (SeqOp2 0 (AggElements () (Int32 +Void) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))) ( (Ref __iruid_1556) (Begin (If (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_1539))) (Let __iruid_1540 (GetField GT (Ref __iruid_1539)) (Let __iruid_1541 (Cast Float64 (GetField DP (Ref __iruid_1539))) (Begin (SeqOp2 0 (Sum () (Int64) None) ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_1540)))))) (SeqOp2 1 (Sum () (Int64) None) ( (Apply toInt64 Int64 (IsNA (Ref __iruid_1540))))) (SeqOp2 2 (Count () () None) ()) (SeqOp2 3 (Sum () (Int64) None) ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_1541)))))) (SeqOp2 4 (Sum () (Float64) None) ( (Ref __iruid_1541)))))) (Begin))))))))))) 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.205ms, total 543.562ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.237ms, total 543.875ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.506ms, total 544.431ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 3.036ms, total 547.518ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.177ms, total 547.758ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 2.562ms, total 550.364ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.304ms, total 550.786ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.156ms, total 551.013ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.346ms, total 551.400ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 2.577ms, total 554.038ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.156ms, total 554.262ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.116ms, total 555.421ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.326ms, total 555.805ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.144ms, total 556.016ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.326ms, total 556.386ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.455ms, total 557.913ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.175ms, total 558.147ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.145ms, total 559.337ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 16.062ms, total 559.414ms, tagged coverage 93.1 2020-01-07 21:47:38 root: INFO: optimize Compile: after: IR size 57: (Begin (Let __iruid_1569 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Begin (SeqOp2 0 (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))) ( (ArrayLen (Ref __iruid_1569)))) (ArrayFor __iruid_1570 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_1569)) (I32 1)) (Let __iruid_1571 (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (ArrayRef (Ref __iruid_1569) (Ref __iruid_1570))) (SeqOp2 0 (AggElements () (Int32 +Void) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))) ( (Ref __iruid_1570) (Begin (If (IsNA (Ref __iruid_1571)) (Begin) (Let __iruid_1572 (GetField GT (Ref __iruid_1571)) (Let __iruid_1573 (Cast Float64 (GetField DP (Ref __iruid_1571))) (Begin (SeqOp2 0 (Sum () (Int64) None) ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_1572)))))) (SeqOp2 1 (Sum () (Int64) None) ( (Apply toInt64 Int64 (IsNA (Ref __iruid_1572))))) (SeqOp2 2 (Count () () None) ()) (SeqOp2 3 (Sum () (Int64) None) ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_1573)))))) (SeqOp2 4 (Sum () (Float64) None) ( (Ref __iruid_1573))))))))))))))) 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C27. 2020-01-07 21:47:38 root: INFO: instruction count: 22: is/hail/codegen/generated/C27.apply 2020-01-07 21:47:38 root: INFO: instruction count: 20: is/hail/codegen/generated/C27.apply 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C27.newAggState 2020-01-07 21:47:38 root: INFO: instruction count: 17: is/hail/codegen/generated/C27.setAggState 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C27.getAggOffset 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C27.setNumSerialized 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C27.setSerializedAgg 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C27.getSerializedAgg 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C27.m7_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 19: is/hail/codegen/generated/C27.m8_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C27.m9_new_state_group0 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C27.m10_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 19: is/hail/codegen/generated/C27.m11_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 53: is/hail/codegen/generated/C27.m12_load_group0 2020-01-07 21:47:38 root: INFO: instruction count: 36: is/hail/codegen/generated/C27.m13_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 17: is/hail/codegen/generated/C27.m14_method 2020-01-07 21:47:38 root: INFO: instruction count: 300: is/hail/codegen/generated/C27.m15_method 2020-01-07 21:47:38 root: INFO: instruction count: 272: is/hail/codegen/generated/C27.m16_method 2020-01-07 21:47:38 root: INFO: instruction count: 249: is/hail/codegen/generated/C27.m17_method 2020-01-07 21:47:38 root: INFO: instruction count: 122: is/hail/codegen/generated/C27.m18_method 2020-01-07 21:47:38 root: INFO: instruction count: 161: is/hail/codegen/generated/C27.m19_method 2020-01-07 21:47:38 root: INFO: instruction count: 5: is/hail/codegen/generated/C27.m20_method 2020-01-07 21:47:38 root: INFO: instruction count: 113: is/hail/codegen/generated/C27.m21_load_group0 2020-01-07 21:47:38 root: INFO: instruction count: 113: is/hail/codegen/generated/C27.m22_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 14: is/hail/codegen/generated/C27.setPartitionIndex 2020-01-07 21:47:38 root: INFO: instruction count: 6: is/hail/codegen/generated/C27.addPartitionRegion 2020-01-07 21:47:38 root: INFO: optimize Compile: before: IR size 1: (DeserializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))))) 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.275ms, total 583.421ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.009ms, total 583.510ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.022ms, total 583.571ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.084ms, total 583.691ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.018ms, total 583.751ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.033ms, total 583.828ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.733ms, total 583.874ms, tagged coverage 60.1 2020-01-07 21:47:38 root: INFO: optimize Compile: after: IR size 1: (DeserializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))))) 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C28. 2020-01-07 21:47:38 root: INFO: instruction count: 37: is/hail/codegen/generated/C28.apply 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C28.apply 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C28.newAggState 2020-01-07 21:47:38 root: INFO: instruction count: 17: is/hail/codegen/generated/C28.setAggState 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C28.getAggOffset 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C28.setNumSerialized 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C28.setSerializedAgg 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C28.getSerializedAgg 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C28.m7_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 19: is/hail/codegen/generated/C28.m8_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C28.m9_new_state_group0 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C28.m10_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 19: is/hail/codegen/generated/C28.m11_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 53: is/hail/codegen/generated/C28.m12_load_group0 2020-01-07 21:47:38 root: INFO: instruction count: 36: is/hail/codegen/generated/C28.m13_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 23: is/hail/codegen/generated/C28.m14_array_nested_deserialize_init_group0 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C28.m15_new_state_group0 2020-01-07 21:47:38 root: INFO: instruction count: 63: is/hail/codegen/generated/C28.m16_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 23: is/hail/codegen/generated/C28.m17_array_nested_deserialize_group0 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C28.m18_new_state_group0 2020-01-07 21:47:38 root: INFO: instruction count: 113: is/hail/codegen/generated/C28.m19_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 110: is/hail/codegen/generated/C28.m20_deserialize_aggs_group0 2020-01-07 21:47:38 root: INFO: instruction count: 14: is/hail/codegen/generated/C28.setPartitionIndex 2020-01-07 21:47:38 root: INFO: instruction count: 6: is/hail/codegen/generated/C28.addPartitionRegion 2020-01-07 21:47:38 root: INFO: optimize Compile: before: IR size 1: (SerializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))))) 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.037ms, total 589.511ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.008ms, total 589.592ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.017ms, total 589.680ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.086ms, total 589.814ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.016ms, total 589.884ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.031ms, total 589.953ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.553ms, total 590.024ms, tagged coverage 35.3 2020-01-07 21:47:38 root: INFO: optimize Compile: after: IR size 1: (SerializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))))) 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C29. 2020-01-07 21:47:38 root: INFO: instruction count: 48: is/hail/codegen/generated/C29.apply 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C29.apply 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C29.newAggState 2020-01-07 21:47:38 root: INFO: instruction count: 17: is/hail/codegen/generated/C29.setAggState 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C29.getAggOffset 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C29.setNumSerialized 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C29.setSerializedAgg 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C29.getSerializedAgg 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C29.m7_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 19: is/hail/codegen/generated/C29.m8_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C29.m9_new_state_group0 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C29.m10_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 19: is/hail/codegen/generated/C29.m11_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 53: is/hail/codegen/generated/C29.m12_load_group0 2020-01-07 21:47:38 root: INFO: instruction count: 36: is/hail/codegen/generated/C29.m13_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 63: is/hail/codegen/generated/C29.m14_load_group0 2020-01-07 21:47:38 root: INFO: instruction count: 23: is/hail/codegen/generated/C29.m15_array_nested_serialize_init_group0 2020-01-07 21:47:38 root: INFO: instruction count: 113: is/hail/codegen/generated/C29.m16_load_group0 2020-01-07 21:47:38 root: INFO: instruction count: 23: is/hail/codegen/generated/C29.m17_array_nested_serialize_group0 2020-01-07 21:47:38 root: INFO: instruction count: 40: is/hail/codegen/generated/C29.m18_serialize_aggs_group0 2020-01-07 21:47:38 root: INFO: instruction count: 36: is/hail/codegen/generated/C29.m19_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 14: is/hail/codegen/generated/C29.setPartitionIndex 2020-01-07 21:47:38 root: INFO: instruction count: 6: is/hail/codegen/generated/C29.addPartitionRegion 2020-01-07 21:47:38 root: INFO: optimize Compile: before: IR size 5: (Begin (DeserializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))))) (DeserializeAggs 1 1 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))))) (CombOp2 0 1 (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None)))) (SerializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None)))))) 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.066ms, total 595.724ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.023ms, total 595.852ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.041ms, total 595.933ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.264ms, total 596.234ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.026ms, total 596.310ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.085ms, total 596.431ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.829ms, total 596.473ms, tagged coverage 60.9 2020-01-07 21:47:38 root: INFO: optimize Compile: after: IR size 5: (Begin (DeserializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))))) (DeserializeAggs 1 1 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))))) (CombOp2 0 1 (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None)))) (SerializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None)))))) 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C30. 2020-01-07 21:47:38 root: INFO: instruction count: 234: is/hail/codegen/generated/C30.apply 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C30.apply 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C30.newAggState 2020-01-07 21:47:38 root: INFO: instruction count: 17: is/hail/codegen/generated/C30.setAggState 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C30.getAggOffset 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C30.setNumSerialized 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C30.setSerializedAgg 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C30.getSerializedAgg 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C30.m7_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C30.m8_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 35: is/hail/codegen/generated/C30.m9_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 11: is/hail/codegen/generated/C30.m10_new_state_group0 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C30.m11_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C30.m12_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 35: is/hail/codegen/generated/C30.m13_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 103: is/hail/codegen/generated/C30.m14_load_group0 2020-01-07 21:47:38 root: INFO: instruction count: 69: is/hail/codegen/generated/C30.m15_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 23: is/hail/codegen/generated/C30.m16_array_nested_deserialize_init_group0 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C30.m17_new_state_group0 2020-01-07 21:47:38 root: INFO: instruction count: 63: is/hail/codegen/generated/C30.m18_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 23: is/hail/codegen/generated/C30.m19_array_nested_deserialize_group0 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C30.m20_new_state_group0 2020-01-07 21:47:38 root: INFO: instruction count: 113: is/hail/codegen/generated/C30.m21_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 110: is/hail/codegen/generated/C30.m22_deserialize_aggs_group0 2020-01-07 21:47:38 root: INFO: instruction count: 23: is/hail/codegen/generated/C30.m23_array_nested_deserialize_init_group0 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C30.m24_new_state_group0 2020-01-07 21:47:38 root: INFO: instruction count: 63: is/hail/codegen/generated/C30.m25_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 23: is/hail/codegen/generated/C30.m26_array_nested_deserialize_group0 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C30.m27_new_state_group0 2020-01-07 21:47:38 root: INFO: instruction count: 113: is/hail/codegen/generated/C30.m28_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 110: is/hail/codegen/generated/C30.m29_deserialize_aggs_group0 2020-01-07 21:47:38 root: INFO: instruction count: 48: is/hail/codegen/generated/C30.m30_copy_from_group0 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C30.m31_new_state_group0 2020-01-07 21:47:38 root: INFO: instruction count: 113: is/hail/codegen/generated/C30.m32_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 113: is/hail/codegen/generated/C30.m33_load_group0 2020-01-07 21:47:38 root: INFO: instruction count: 113: is/hail/codegen/generated/C30.m34_load_group0 2020-01-07 21:47:38 root: INFO: instruction count: 38: is/hail/codegen/generated/C30.m35_array_nested_comb_group0 2020-01-07 21:47:38 root: INFO: instruction count: 113: is/hail/codegen/generated/C30.m36_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 63: is/hail/codegen/generated/C30.m37_load_group0 2020-01-07 21:47:38 root: INFO: instruction count: 23: is/hail/codegen/generated/C30.m38_array_nested_serialize_init_group0 2020-01-07 21:47:38 root: INFO: instruction count: 113: is/hail/codegen/generated/C30.m39_load_group0 2020-01-07 21:47:38 root: INFO: instruction count: 23: is/hail/codegen/generated/C30.m40_array_nested_serialize_group0 2020-01-07 21:47:38 root: INFO: instruction count: 40: is/hail/codegen/generated/C30.m41_serialize_aggs_group0 2020-01-07 21:47:38 root: INFO: instruction count: 69: is/hail/codegen/generated/C30.m42_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 14: is/hail/codegen/generated/C30.setPartitionIndex 2020-01-07 21:47:38 root: INFO: instruction count: 6: is/hail/codegen/generated/C30.addPartitionRegion 2020-01-07 21:47:38 root: INFO: optimize Compile: before: IR size 29: (Let __iruid_1553 (ResultOp2 0 ( (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))))) (MakeTuple (0) (Let __iruid_1557 (GetTupleElement 0 (Ref __iruid_1553)) (ArrayMap __iruid_1538 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_1557)) (I32 1)) (Let __iruid_1554 (ArrayRef (Ref __iruid_1557) (Ref __iruid_1538)) (MakeStruct (__iruid_1445 (MakeStruct (__iruid_1439 (GetTupleElement 0 (Ref __iruid_1554))) (__iruid_1440 (GetTupleElement 1 (Ref __iruid_1554))) (__iruid_1441 (GetTupleElement 2 (Ref __iruid_1554))) (__iruid_1444 (MakeStruct (__iruid_1442 (GetTupleElement 3 (Ref __iruid_1554))) (__iruid_1443 (GetTupleElement 4 (Ref __iruid_1554))))))))))))) 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.450ms, total 607.266ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.100ms, total 607.444ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.230ms, total 607.723ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.563ms, total 609.330ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.103ms, total 609.491ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.479ms, total 611.012ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.241ms, total 611.310ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.092ms, total 611.454ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.189ms, total 611.684ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.799ms, total 612.521ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.127ms, total 612.703ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.919ms, total 613.664ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.163ms, total 613.882ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.079ms, total 614.007ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.156ms, total 614.201ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.387ms, total 615.624ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.112ms, total 615.796ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.863ms, total 616.696ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 9.944ms, total 616.755ms, tagged coverage 91.0 2020-01-07 21:47:38 root: INFO: optimize Compile: after: IR size 29: (Let __iruid_1582 (ResultOp2 0 ( (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))))) (MakeTuple (0) (Let __iruid_1583 (GetTupleElement 0 (Ref __iruid_1582)) (ArrayMap __iruid_1584 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_1583)) (I32 1)) (Let __iruid_1585 (ArrayRef (Ref __iruid_1583) (Ref __iruid_1584)) (MakeStruct (__iruid_1445 (MakeStruct (__iruid_1439 (GetTupleElement 0 (Ref __iruid_1585))) (__iruid_1440 (GetTupleElement 1 (Ref __iruid_1585))) (__iruid_1441 (GetTupleElement 2 (Ref __iruid_1585))) (__iruid_1444 (MakeStruct (__iruid_1442 (GetTupleElement 3 (Ref __iruid_1585))) (__iruid_1443 (GetTupleElement 4 (Ref __iruid_1585))))))))))))) 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C31. 2020-01-07 21:47:38 root: INFO: instruction count: 84: is/hail/codegen/generated/C31.apply 2020-01-07 21:47:38 root: INFO: instruction count: 16: is/hail/codegen/generated/C31.apply 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C31.newAggState 2020-01-07 21:47:38 root: INFO: instruction count: 17: is/hail/codegen/generated/C31.setAggState 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C31.getAggOffset 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C31.setNumSerialized 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C31.setSerializedAgg 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C31.getSerializedAgg 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C31.m7_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 19: is/hail/codegen/generated/C31.m8_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C31.m9_new_state_group0 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C31.m10_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 19: is/hail/codegen/generated/C31.m11_create_states_group0 2020-01-07 21:47:38 root: INFO: instruction count: 53: is/hail/codegen/generated/C31.m12_load_group0 2020-01-07 21:47:38 root: INFO: instruction count: 36: is/hail/codegen/generated/C31.m13_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 113: is/hail/codegen/generated/C31.m14_load_group0 2020-01-07 21:47:38 root: INFO: instruction count: 57: is/hail/codegen/generated/C31.m15_array_nested_result_group0 2020-01-07 21:47:38 root: INFO: instruction count: 137: is/hail/codegen/generated/C31.m16_aggs_result_group0 2020-01-07 21:47:38 root: INFO: instruction count: 36: is/hail/codegen/generated/C31.m17_store_group0 2020-01-07 21:47:38 root: INFO: instruction count: 486: is/hail/codegen/generated/C31.m18_method 2020-01-07 21:47:38 root: INFO: instruction count: 46: is/hail/codegen/generated/C31.m19_method 2020-01-07 21:47:38 root: INFO: instruction count: 275: is/hail/codegen/generated/C31.m20_method 2020-01-07 21:47:38 root: INFO: instruction count: 14: is/hail/codegen/generated/C31.setPartitionIndex 2020-01-07 21:47:38 root: INFO: instruction count: 6: is/hail/codegen/generated/C31.addPartitionRegion 2020-01-07 21:47:38 root: INFO: Aggregate: useTreeAggregate=true 2020-01-07 21:47:38 root: INFO: Aggregate: commutative=true 2020-01-07 21:47:38 SparkContext: INFO: Starting job: runJob at RVD.scala:605 2020-01-07 21:47:38 DAGScheduler: INFO: Got job 6 (runJob at RVD.scala:605) with 2 output partitions 2020-01-07 21:47:38 DAGScheduler: INFO: Final stage: ResultStage 6 (runJob at RVD.scala:605) 2020-01-07 21:47:38 DAGScheduler: INFO: Parents of final stage: List() 2020-01-07 21:47:38 DAGScheduler: INFO: Missing parents: List() 2020-01-07 21:47:38 DAGScheduler: INFO: Submitting ResultStage 6 (MapPartitionsRDD[32] at mapPartitions at ContextRDD.scala:217), which has no missing parents 2020-01-07 21:47:38 MemoryStore: INFO: Block broadcast_15 stored as values in memory (estimated size 71.6 KB, free 365.6 MB) 2020-01-07 21:47:38 MemoryStore: INFO: Block broadcast_15_piece0 stored as bytes in memory (estimated size 33.8 KB, free 365.6 MB) 2020-01-07 21:47:38 BlockManagerInfo: INFO: Added broadcast_15_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 33.8 KB, free: 366.2 MB) 2020-01-07 21:47:38 SparkContext: INFO: Created broadcast 15 from broadcast at DAGScheduler.scala:1161 2020-01-07 21:47:38 DAGScheduler: INFO: Submitting 2 missing tasks from ResultStage 6 (MapPartitionsRDD[32] at mapPartitions at ContextRDD.scala:217) (first 15 tasks are for partitions Vector(0, 1)) 2020-01-07 21:47:38 TaskSchedulerImpl: INFO: Adding task set 6.0 with 2 tasks 2020-01-07 21:47:38 TaskSetManager: INFO: Starting task 0.0 in stage 6.0 (TID 40, localhost, executor driver, partition 0, PROCESS_LOCAL, 8296 bytes) 2020-01-07 21:47:38 TaskSetManager: INFO: Starting task 1.0 in stage 6.0 (TID 41, localhost, executor driver, partition 1, PROCESS_LOCAL, 8296 bytes) 2020-01-07 21:47:38 Executor: INFO: Running task 0.0 in stage 6.0 (TID 40) 2020-01-07 21:47:38 Executor: INFO: Running task 1.0 in stage 6.0 (TID 41) 2020-01-07 21:47:38 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:38 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:38 root: INFO: RegionPool: initialized for thread 226: Executor task launch worker for task 41 2020-01-07 21:47:38 root: INFO: RegionPool: REPORT_THRESHOLD: 317.1K allocated (216.0K blocks / 101.1K chunks), thread 226: Executor task launch worker for task 41 2020-01-07 21:47:38 root: INFO: RegionPool: REPORT_THRESHOLD: 314.6K allocated (216.0K blocks / 98.6K chunks), thread 225: Executor task launch worker for task 39 2020-01-07 21:47:38 root: INFO: RegionPool: REPORT_THRESHOLD: 549.3K allocated (280.0K blocks / 269.3K chunks), thread 225: Executor task launch worker for task 39 2020-01-07 21:47:38 root: INFO: RegionPool: REPORT_THRESHOLD: 549.3K allocated (280.0K blocks / 269.3K chunks), thread 226: Executor task launch worker for task 41 2020-01-07 21:47:38 root: INFO: Index reader cache queries: 8 2020-01-07 21:47:38 root: INFO: Index reader cache hit rate: 0.625 2020-01-07 21:47:38 Executor: INFO: Finished task 1.0 in stage 6.0 (TID 41). 12135 bytes result sent to driver 2020-01-07 21:47:38 TaskSetManager: INFO: Finished task 1.0 in stage 6.0 (TID 41) in 349 ms on localhost (executor driver) (1/2) 2020-01-07 21:47:38 root: INFO: Index reader cache queries: 8 2020-01-07 21:47:38 root: INFO: Index reader cache hit rate: 0.625 2020-01-07 21:47:38 Executor: INFO: Finished task 0.0 in stage 6.0 (TID 40). 12135 bytes result sent to driver 2020-01-07 21:47:38 TaskSetManager: INFO: Finished task 0.0 in stage 6.0 (TID 40) in 360 ms on localhost (executor driver) (2/2) 2020-01-07 21:47:38 TaskSchedulerImpl: INFO: Removed TaskSet 6.0, whose tasks have all completed, from pool 2020-01-07 21:47:38 DAGScheduler: INFO: ResultStage 6 (runJob at RVD.scala:605) finished in 0.373 s 2020-01-07 21:47:38 DAGScheduler: INFO: Job 6 finished: runJob at RVD.scala:605, took 0.377873 s 2020-01-07 21:47:38 root: INFO: took 585.538ms 2020-01-07 21:47:38 root: INFO: interpreting non compilable node: Literal 2020-01-07 21:47:38 root: INFO: took 0.052ms 2020-01-07 21:47:38 root: INFO: interpreting non compilable node: TableCollect 2020-01-07 21:47:38 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:38 root: INFO: decoder cache hit 2020-01-07 21:47:38 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:38 root: INFO: decoder cache hit 2020-01-07 21:47:38 root: INFO: encoder cache hit 2020-01-07 21:47:38 SparkContext: INFO: Starting job: collect at RVD.scala:676 2020-01-07 21:47:38 DAGScheduler: INFO: Got job 7 (collect at RVD.scala:676) with 1 output partitions 2020-01-07 21:47:38 DAGScheduler: INFO: Final stage: ResultStage 7 (collect at RVD.scala:676) 2020-01-07 21:47:38 DAGScheduler: INFO: Parents of final stage: List() 2020-01-07 21:47:38 DAGScheduler: INFO: Missing parents: List() 2020-01-07 21:47:38 DAGScheduler: INFO: Submitting ResultStage 7 (MapPartitionsRDD[38] at mapPartitions at ContextRDD.scala:217), which has no missing parents 2020-01-07 21:47:38 MemoryStore: INFO: Block broadcast_16 stored as values in memory (estimated size 10.3 KB, free 365.6 MB) 2020-01-07 21:47:38 MemoryStore: INFO: Block broadcast_16_piece0 stored as bytes in memory (estimated size 5.5 KB, free 365.6 MB) 2020-01-07 21:47:38 BlockManagerInfo: INFO: Added broadcast_16_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 5.5 KB, free: 366.2 MB) 2020-01-07 21:47:38 SparkContext: INFO: Created broadcast 16 from broadcast at DAGScheduler.scala:1161 2020-01-07 21:47:38 DAGScheduler: INFO: Submitting 1 missing tasks from ResultStage 7 (MapPartitionsRDD[38] at mapPartitions at ContextRDD.scala:217) (first 15 tasks are for partitions Vector(0)) 2020-01-07 21:47:38 TaskSchedulerImpl: INFO: Adding task set 7.0 with 1 tasks 2020-01-07 21:47:38 TaskSetManager: INFO: Starting task 0.0 in stage 7.0 (TID 42, localhost, executor driver, partition 0, PROCESS_LOCAL, 7653 bytes) 2020-01-07 21:47:38 Executor: INFO: Running task 0.0 in stage 7.0 (TID 42) 2020-01-07 21:47:38 Executor: INFO: Finished task 0.0 in stage 7.0 (TID 42). 3940 bytes result sent to driver 2020-01-07 21:47:38 TaskSetManager: INFO: Finished task 0.0 in stage 7.0 (TID 42) in 8 ms on localhost (executor driver) (1/1) 2020-01-07 21:47:38 TaskSchedulerImpl: INFO: Removed TaskSet 7.0, whose tasks have all completed, from pool 2020-01-07 21:47:38 DAGScheduler: INFO: ResultStage 7 (collect at RVD.scala:676) finished in 0.019 s 2020-01-07 21:47:38 DAGScheduler: INFO: Job 7 finished: collect at RVD.scala:676, took 0.021839 s 2020-01-07 21:47:38 root: INFO: decoder cache hit 2020-01-07 21:47:38 root: INFO: took 58.014ms 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation : 717.891ms, total 1.087s, tagged coverage 4.0 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- Verify : 0.221ms, total 1.087s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable : 718.507ms, total 1.087s, tagged coverage 4.0 2020-01-07 21:47:38 root: INFO: optimize InterpretNonCompilable, post Lowering: before: IR size 91: (Cast Int64 (ArrayLen (GetField rows (Let __iruid_1542 (GetField rows (Literal Struct{rows:Array[Struct{}],global:Struct{}} )) (Let __iruid_1543 (ArrayMap __iruid_1544 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_1542)) (I32 1)) (Let __iruid_1545 (GetField __iruid_1445 (ArrayRef (Literal Array[Struct{__iruid_1445:Struct{__iruid_1439:Int64,__iruid_1440:Int64,__iruid_1441:Int64,__iruid_1444:Struct{__iruid_1442:Int64,__iruid_1443:Float64}}}] ) (Ref __iruid_1544))) (Let __iruid_1546 (GetField __iruid_1444 (Ref __iruid_1545)) (InsertFields (ArrayRef (Ref __iruid_1542) (Ref __iruid_1544)) None (sample_qc (Let __iruid_1547 (MakeStruct (n_called (GetField __iruid_1439 (Ref __iruid_1545))) (n_not_called (GetField __iruid_1440 (Ref __iruid_1545))) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (GetField __iruid_1441 (Ref __iruid_1545))))) (Let __iruid_1548 (GetField n_called (Ref __iruid_1547)) (MakeStruct (dp_stats (Let __iruid_1549 (MakeStruct (n_def (GetField __iruid_1442 (Ref __iruid_1546))) (sum (GetField __iruid_1443 (Ref __iruid_1546)))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_1549)) (Cast Float64 (GetField n_def (Ref __iruid_1549)))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (Ref __iruid_1548)) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (Ref __iruid_1548) (GetField n_not_called (Ref __iruid_1547))) (GetField n_filtered (Ref __iruid_1547)))))))))))))) (MakeStruct (rows (ArrayMap __iruid_1550 (ArrayFilter __iruid_1551 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_1543)) (I32 1)) (Let __iruid_1552 (GetField sample_qc (ArrayRef (Ref __iruid_1543) (Ref __iruid_1551))) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (Ref __iruid_1552))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (Ref __iruid_1552)) (F64 0.97))) (False)))) (ArrayRef (Ref __iruid_1543) (Ref __iruid_1550)))))))))) 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.974ms, total 1.090s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 1.100ms, total 1.091s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.871ms, total 1.092s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 4.515ms, total 1.096s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.515ms, total 1.097s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 3.408ms, total 1.100s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.614ms, total 1.101s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.167ms, total 1.101s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.785ms, total 1.102s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 2.755ms, total 1.105s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.348ms, total 1.105s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 2.648ms, total 1.108s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.491ms, total 1.109s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.173ms, total 1.109s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.748ms, total 1.110s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 3.543ms, total 1.113s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.362ms, total 1.114s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 2.532ms, total 1.116s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 27.967ms, total 1.117s, tagged coverage 94.9 2020-01-07 21:47:38 root: INFO: optimize InterpretNonCompilable, post Lowering: after: IR size 87: (Cast Int64 (ArrayLen (GetField rows (Let __iruid_1607 (ArrayMap __iruid_1608 (ArrayRange (I32 0) (I32 284) (I32 1)) (Let __iruid_1609 (GetField __iruid_1445 (ArrayRef (Literal Array[Struct{__iruid_1445:Struct{__iruid_1439:Int64,__iruid_1440:Int64,__iruid_1441:Int64,__iruid_1444:Struct{__iruid_1442:Int64,__iruid_1443:Float64}}}] ) (Ref __iruid_1608))) (Let __iruid_1610 (GetField __iruid_1444 (Ref __iruid_1609)) (InsertFields (ArrayRef (Literal Array[Struct{}] ) (Ref __iruid_1608)) None (sample_qc (Let __iruid_1611 (MakeStruct (n_called (GetField __iruid_1439 (Ref __iruid_1609))) (n_not_called (GetField __iruid_1440 (Ref __iruid_1609))) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (GetField __iruid_1441 (Ref __iruid_1609))))) (Let __iruid_1612 (GetField n_called (Ref __iruid_1611)) (MakeStruct (dp_stats (Let __iruid_1613 (MakeStruct (n_def (GetField __iruid_1442 (Ref __iruid_1610))) (sum (GetField __iruid_1443 (Ref __iruid_1610)))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_1613)) (Cast Float64 (GetField n_def (Ref __iruid_1613)))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (Ref __iruid_1612)) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (Ref __iruid_1612) (GetField n_not_called (Ref __iruid_1611))) (GetField n_filtered (Ref __iruid_1611)))))))))))))) (MakeStruct (rows (ArrayMap __iruid_1614 (ArrayFilter __iruid_1615 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_1607)) (I32 1)) (Let __iruid_1616 (GetField sample_qc (ArrayRef (Ref __iruid_1607) (Ref __iruid_1615))) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (Ref __iruid_1616))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (Ref __iruid_1616)) (F64 0.97))) (False)))) (ArrayRef (Ref __iruid_1607) (Ref __iruid_1614))))))))) 2020-01-07 21:47:38 root: INFO: encoder cache miss (4 hits, 10 misses, 0.286) 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_33. 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_33.apply 2020-01-07 21:47:38 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_33.m0_ENCODE_o_tuple_of_o_array_of_o_tuple_of_ENDANDo_array_of_o_tuple_of_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDENDENDEND_TO_o_struct_of_o_array_of_o_struct_of_ENDANDo_array_of_o_struct_of_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDENDENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 39: is/hail/codegen/generated/C_etypeEncode_33.m1_write_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 74: is/hail/codegen/generated/C_etypeEncode_33.m2_ENCODE_o_array_of_o_tuple_of_END_TO_o_array_of_o_struct_of_END 2020-01-07 21:47:38 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_33.m3_ENCODE_o_tuple_of_END_TO_o_struct_of_END 2020-01-07 21:47:38 root: INFO: instruction count: 74: is/hail/codegen/generated/C_etypeEncode_33.m4_ENCODE_o_array_of_o_tuple_of_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDENDEND_TO_o_array_of_o_struct_of_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_33.m5_ENCODE_o_tuple_of_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDENDEND_TO_o_struct_of_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 19: is/hail/codegen/generated/C_etypeEncode_33.m6_write_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_33.m7_ENCODE_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDEND_TO_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 73: is/hail/codegen/generated/C_etypeEncode_33.m8_write_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_33.m9_ENCODE_o_int64_TO_o_int64 2020-01-07 21:47:38 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_33.m10_ENCODE_o_tuple_of_o_int64ANDo_float64END_TO_o_struct_of_o_int64ANDo_float64END 2020-01-07 21:47:38 root: INFO: instruction count: 39: is/hail/codegen/generated/C_etypeEncode_33.m11_write_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_33.m12_ENCODE_o_float64_TO_o_float64 2020-01-07 21:47:38 MemoryStore: INFO: Block broadcast_17 stored as values in memory (estimated size 12.1 KB, free 365.6 MB) 2020-01-07 21:47:38 MemoryStore: INFO: Block broadcast_17_piece0 stored as bytes in memory (estimated size 5.3 KB, free 365.6 MB) 2020-01-07 21:47:38 BlockManagerInfo: INFO: Added broadcast_17_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 5.3 KB, free: 366.2 MB) 2020-01-07 21:47:38 SparkContext: INFO: Created broadcast 17 from broadcast at SparkBackend.scala:21 2020-01-07 21:47:38 root: INFO: instruction count: 3: is/hail/codegen/generated/C32. 2020-01-07 21:47:38 root: INFO: instruction count: 39: is/hail/codegen/generated/C32.apply 2020-01-07 21:47:38 root: INFO: instruction count: 10: is/hail/codegen/generated/C32.apply 2020-01-07 21:47:38 root: INFO: instruction count: 72: is/hail/codegen/generated/C32.m1_method 2020-01-07 21:47:38 root: INFO: instruction count: 607: is/hail/codegen/generated/C32.m2_method 2020-01-07 21:47:38 root: INFO: instruction count: 95: is/hail/codegen/generated/C32.m3_method 2020-01-07 21:47:38 root: INFO: instruction count: 41: is/hail/codegen/generated/C32.m4_method 2020-01-07 21:47:38 root: INFO: instruction count: 147: is/hail/codegen/generated/C32.m5_method 2020-01-07 21:47:38 root: INFO: instruction count: 144: is/hail/codegen/generated/C32.m6_method 2020-01-07 21:47:38 root: INFO: instruction count: 128: is/hail/codegen/generated/C32.m7_method 2020-01-07 21:47:38 root: INFO: instruction count: 77: is/hail/codegen/generated/C32.m8_method 2020-01-07 21:47:38 root: INFO: instruction count: 964: is/hail/codegen/generated/C32.m9_method 2020-01-07 21:47:38 root: INFO: instruction count: 39: is/hail/codegen/generated/C32.m10_method 2020-01-07 21:47:38 root: INFO: instruction count: 14: is/hail/codegen/generated/C32.setPartitionIndex 2020-01-07 21:47:38 root: INFO: instruction count: 6: is/hail/codegen/generated/C32.addPartitionRegion 2020-01-07 21:47:38 root: INFO: instruction count: 27: is/hail/codegen/generated/C32.m13_DECODE_o_struct_of_o_array_of_o_struct_of_ENDANDo_array_of_o_struct_of_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDENDENDEND_TO_o_tuple_of_o_array_of_o_tuple_of_ENDANDo_array_of_o_tuple_of_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDENDENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 53: is/hail/codegen/generated/C32.m14_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 97: is/hail/codegen/generated/C32.m15_INPLACE_DECODE_o_array_of_o_struct_of_END_TO_o_array_of_o_tuple_of_END 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C32.m16_INPLACE_DECODE_o_struct_of_END_TO_o_tuple_of_END 2020-01-07 21:47:38 root: INFO: instruction count: 97: is/hail/codegen/generated/C32.m17_INPLACE_DECODE_o_array_of_o_struct_of_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDENDEND_TO_o_array_of_o_tuple_of_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C32.m18_INPLACE_DECODE_o_struct_of_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDENDEND_TO_o_tuple_of_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 28: is/hail/codegen/generated/C32.m19_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C32.m20_INPLACE_DECODE_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDEND_TO_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDEND 2020-01-07 21:47:38 root: INFO: instruction count: 103: is/hail/codegen/generated/C32.m21_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C32.m22_INPLACE_DECODE_o_int64_TO_o_int64 2020-01-07 21:47:38 root: INFO: instruction count: 21: is/hail/codegen/generated/C32.m23_INPLACE_DECODE_o_struct_of_o_int64ANDo_float64END_TO_o_tuple_of_o_int64ANDo_float64END 2020-01-07 21:47:38 root: INFO: instruction count: 53: is/hail/codegen/generated/C32.m24_read_fields_group_0 2020-01-07 21:47:38 root: INFO: instruction count: 8: is/hail/codegen/generated/C32.m25_INPLACE_DECODE_o_float64_TO_o_float64 2020-01-07 21:47:38 root: INFO: instruction count: 37: is/hail/codegen/generated/C32.addLiterals 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- Compile : 57.474ms, total 1.175s 2020-01-07 21:47:38 root: INFO: Timer: Time taken for CompileAndEvaluate -- InitializeCompiledFunction : 2.848ms, total 1.178s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- RunCompiledFunction : 2.490ms, total 1.181s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- ConvertToSafeValue : 0.165ms, total 1.181s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate : 1.181s, total 1.181s, tagged coverage 39.0 2020-01-07 21:47:39 root: INFO: Timer: all timings: 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 4.105ms, total 12.380ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 19.222ms, total 31.782ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 20.300ms, total 52.180ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 52.319ms, total 104.610ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 3.377ms, total 108.127ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 46.858ms, total 155.084ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 1.648ms, total 156.892ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 9.291ms, total 166.288ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 3.561ms, total 169.984ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 32.883ms, total 202.978ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 1.434ms, total 204.558ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 5.552ms, total 210.193ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.342ms, total 210.664ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.337ms, total 211.059ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 3.981ms, total 215.100ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 3.187ms, total 218.355ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 1.305ms, total 219.730ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 3.588ms, total 223.374ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 215.168ms, total 223.436ms, tagged coverage 99.1 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- Verify : 0.095ms, total 225.135ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- LoweringTransformation : 35.910ms, total 261.125ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- Verify : 1.217ms, total 262.442ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable : 37.474ms, total 262.511ms, tagged coverage 99.3 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 1.292ms, total 273.853ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 1.220ms, total 275.133ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 34.845ms, total 310.031ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 8.893ms, total 319.010ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 1.461ms, total 320.544ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 18.440ms, total 339.042ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.490ms, total 339.606ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.409ms, total 340.081ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.985ms, total 341.115ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 5.667ms, total 346.833ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.475ms, total 347.361ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 4.885ms, total 352.310ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.692ms, total 353.194ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.341ms, total 353.599ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.914ms, total 354.581ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 5.549ms, total 360.213ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.531ms, total 360.820ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 4.443ms, total 365.315ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 92.816ms, total 365.371ms, tagged coverage 98.6 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- Verify : 0.119ms, total 368.869ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.103ms, total 475.395ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.103ms, total 475.584ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.069ms, total 475.725ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.317ms, total 476.090ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.044ms, total 476.197ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.541ms, total 476.786ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 1.589ms, total 476.876ms, tagged coverage 74.0 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.208ms, total 506.713ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.030ms, total 506.819ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.044ms, total 506.907ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.253ms, total 507.223ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.046ms, total 507.325ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.179ms, total 507.549ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.112ms, total 507.717ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.041ms, total 507.840ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.031ms, total 507.909ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.152ms, total 508.096ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.037ms, total 508.179ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.144ms, total 508.377ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 1.926ms, total 508.425ms, tagged coverage 66.4 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.088ms, total 533.108ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.055ms, total 533.227ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.080ms, total 533.351ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.395ms, total 533.785ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.048ms, total 533.881ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.224ms, total 534.145ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 1.195ms, total 534.209ms, tagged coverage 74.5 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.205ms, total 543.562ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.237ms, total 543.875ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.506ms, total 544.431ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 3.036ms, total 547.518ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.177ms, total 547.758ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 2.562ms, total 550.364ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.304ms, total 550.786ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.156ms, total 551.013ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.346ms, total 551.400ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 2.577ms, total 554.038ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.156ms, total 554.262ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.116ms, total 555.421ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.326ms, total 555.805ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.144ms, total 556.016ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.326ms, total 556.386ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.455ms, total 557.913ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.175ms, total 558.147ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.145ms, total 559.337ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 16.062ms, total 559.414ms, tagged coverage 93.1 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.275ms, total 583.421ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.009ms, total 583.510ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.022ms, total 583.571ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.084ms, total 583.691ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.018ms, total 583.751ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.033ms, total 583.828ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.733ms, total 583.874ms, tagged coverage 60.1 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.037ms, total 589.511ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.008ms, total 589.592ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.017ms, total 589.680ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.086ms, total 589.814ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.016ms, total 589.884ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.031ms, total 589.953ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.553ms, total 590.024ms, tagged coverage 35.3 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.066ms, total 595.724ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.023ms, total 595.852ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.041ms, total 595.933ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.264ms, total 596.234ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.026ms, total 596.310ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.085ms, total 596.431ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.829ms, total 596.473ms, tagged coverage 60.9 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.450ms, total 607.266ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.100ms, total 607.444ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.230ms, total 607.723ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.563ms, total 609.330ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.103ms, total 609.491ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.479ms, total 611.012ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.241ms, total 611.310ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.092ms, total 611.454ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.189ms, total 611.684ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.799ms, total 612.521ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.127ms, total 612.703ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.919ms, total 613.664ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.163ms, total 613.882ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.079ms, total 614.007ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.156ms, total 614.201ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.387ms, total 615.624ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.112ms, total 615.796ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.863ms, total 616.696ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 9.944ms, total 616.755ms, tagged coverage 91.0 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation : 717.891ms, total 1.087s, tagged coverage 4.0 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- Verify : 0.221ms, total 1.087s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable : 718.507ms, total 1.087s, tagged coverage 4.0 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.974ms, total 1.090s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 1.100ms, total 1.091s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.871ms, total 1.092s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 4.515ms, total 1.096s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.515ms, total 1.097s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 3.408ms, total 1.100s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.614ms, total 1.101s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.167ms, total 1.101s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.785ms, total 1.102s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 2.755ms, total 1.105s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.348ms, total 1.105s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 2.648ms, total 1.108s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.491ms, total 1.109s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.173ms, total 1.109s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.748ms, total 1.110s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 3.543ms, total 1.113s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.362ms, total 1.114s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 2.532ms, total 1.116s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 27.967ms, total 1.117s, tagged coverage 94.9 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- Compile : 57.474ms, total 1.175s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- InitializeCompiledFunction : 2.848ms, total 1.178s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- RunCompiledFunction : 2.490ms, total 1.181s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate -- ConvertToSafeValue : 0.165ms, total 1.181s 2020-01-07 21:47:39 root: INFO: Timer: Time taken for CompileAndEvaluate : 1.181s, total 1.181s, tagged coverage 39.0 2020-01-07 21:47:39 root: INFO: Timer: aggregate: 2020-01-07 21:47:39 root: INFO: Timer: Time taken for tag 'ConvertToSafeValue' (1): 0.165ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for tag 'Verify' (4): 1.653ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for tag 'RunCompiledFunction' (1): 2.490ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for tag 'InitializeCompiledFunction' (1): 2.848ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for tag 'ForwardRelationalLets' (22): 10.894ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for tag 'FoldConstants' (22): 13.225ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for tag 'ExtractIntervalFilters' (22): 33.339ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for tag 'LoweringTransformation' (1): 35.910ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for tag 'Compile' (1): 57.474ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for tag 'Simplify' (22): 69.050ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for tag 'PruneDeadFields' (22): 101.674ms 2020-01-07 21:47:39 root: INFO: Timer: Time taken for tag 'ForwardLets' (22): 131.679ms 2020-01-07 21:47:39 root: INFO: Timer: Fraction covered by a tagged leaf: 720.931ms (39.0%) 2020-01-07 21:47:47 root: INFO: optimize Lowerer, initial IR: before: IR size 535: (MatrixAggregate (MatrixFilterCols (MatrixMapRows (MatrixMapCols None (MatrixMapRows (MatrixMapCols None (MatrixAnnotateColsTable "__uid_3" (MatrixRead None False False "{\"name\":\"MatrixNativeReader\",\"path\":\"data/1kg.mt\",\"_spec\":null}") (TableKeyBy (Sample) False (TableRead None False "{\"name\":\"TextTableReader\",\"options\":{\"files\":[\"data/1kg_annotations.txt\"],\"typeMapStr\":{},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"impute\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"forceGZ\":false}}"))) (InsertFields (SelectFields (s) (Ref sa)) None (pheno (GetField __uid_3 (Ref sa))))) (InsertFields (SelectFields (locus alleles rsid qual filters info) (Ref va)) None (__uid_4 (GetField AC (ApplyAggOp CallStats ( (ArrayLen (GetField alleles (Ref va)))) ( (GetField GT (Ref g)))))) (__uid_5 (ArrayMap __uid_6 (ApplyIR `[*:]` Array[String] (GetField alleles (Ref va)) (I32 1)) (Let __uid_11 (ApplyIR __uid_1 Int32 (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (Ref __uid_6)) (If (ApplyComparisonOp EQ (Ref __uid_11) (I32 1)) (If (ApplySpecial `&&` Boolean (ApplyComparisonOp EQ (ApplyIR __uid_1 Int32 (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (Ref __uid_6)) (I32 1)) (ApplyIR __uid_2 Boolean (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (Ref __uid_6))) (I32 8) (I32 9)) (Ref __uid_11))))))) (AggLet __cse_3 False (GetField GT (Ref g)) (Let __cse_16 (ApplyIR toInt64 Int64 (I32 0)) (InsertFields (SelectFields (s pheno) (Ref sa)) None (sample_qc (Let __uid_27 (MakeStruct (n_called (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __cse_3))))))) (n_not_called (ApplyAggOp Sum () ( (Apply toInt64 Int64 (IsNA (Ref __cse_3)))))) (n_filtered (ApplyBinaryPrimOp Subtract (TableCount (MatrixRowsTable (MatrixMapRows (MatrixMapCols None (MatrixAnnotateColsTable "__uid_3" (MatrixRead None False False "{\"name\":\"MatrixNativeReader\",\"path\":\"data/1kg.mt\",\"_spec\":null}") (TableKeyBy (Sample) False (TableRead None False "{\"name\":\"TextTableReader\",\"options\":{\"files\":[\"data/1kg_annotations.txt\"],\"typeMapStr\":{},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"impute\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"forceGZ\":false}}"))) (InsertFields (SelectFields (s) (Ref sa)) None (pheno (GetField __uid_3 (Ref sa))))) (Let __cse_4 (GetField alleles (Ref va)) (InsertFields (SelectFields (locus alleles rsid qual filters info) (Ref va)) None (__uid_4 (GetField AC (ApplyAggOp CallStats ( (ArrayLen (GetField alleles (Ref va)))) ( (GetField GT (Ref g)))))) (__uid_5 (ArrayMap __uid_6 (ApplyIR `[*:]` Array[String] (Ref __cse_4) (I32 1)) (Let __uid_11 (ApplyIR __uid_1 Int32 (ApplyIR indexArray String (Ref __cse_4) (I32 0)) (Ref __uid_6)) (If (ApplyComparisonOp EQ (Ref __uid_11) (I32 1)) (Let __cse_5 (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (If (ApplySpecial `&&` Boolean (ApplyComparisonOp EQ (ApplyIR __uid_1 Int32 (Ref __cse_5) (Ref __uid_6)) (I32 1)) (ApplyIR __uid_2 Boolean (Ref __cse_5) (Ref __uid_6))) (I32 8) (I32 9))) (Ref __uid_11)))))))))) (ApplyAggOp Count () ()))) (n_hom_ref (ApplyAggOp Sum () ( (Apply toInt64 Int64 (Apply isHomRef Boolean (Ref __cse_3)))))) (n_het (ApplyAggOp Sum () ( (Apply toInt64 Int64 (Apply isHet Boolean (Ref __cse_3)))))) (n_singleton (ApplyAggOp Sum () ( (ApplyIR toInt64 Int64 (ApplyIR sum Int32 (ArrayFilter __uid_24 (ArrayMap __uid_23 (ArrayMap __uid_22 (ArrayRange (I32 0) (Apply ploidy Int32 (Ref __cse_3)) (I32 1)) (ApplyComparisonOp EQ (ApplyIR indexArray Int32 (GetField __uid_4 (Ref va)) (Apply `[]` Int32 (Ref __cse_3) (Ref __uid_22))) (I32 1))) (Apply toInt32 Int32 (Ref __uid_23))) (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_24))))))))) (allele_type_counts (AggExplode __uid_26 False (ArrayMap __uid_25 (ArrayRange (I32 0) (Apply ploidy Int32 (Ref __cse_3)) (I32 1)) (If (ApplyComparisonOp GT (Apply `[]` Int32 (Ref __cse_3) (Ref __uid_25)) (I32 0)) (ApplyIR indexArray Int32 (GetField __uid_5 (Ref va)) (ApplyBinaryPrimOp Subtract (Apply `[]` Int32 (GetField GT (Ref g)) (Ref __uid_25)) (I32 1))) (NA Int32))) (AggGroupBy False (Ref __uid_26) (ApplyAggOp Count () ()))))) (Let __cse_10 (GetField n_called (Ref __uid_27)) (Let __cse_11 (GetField n_not_called (Ref __uid_27)) (Let __cse_12 (GetField n_filtered (Ref __uid_27)) (Let __cse_13 (GetField n_hom_ref (Ref __uid_27)) (Let __cse_14 (GetField n_het (Ref __uid_27)) (Let __cse_15 (GetField allele_type_counts (Ref __uid_27)) (Let __uid_28 (MakeStruct (dp_stats (SelectFields (mean stdev min max) (AggLet __uid_agg16 False (ApplyIR toFloat64 Float64 (GetField DP (Ref g))) (Let __uid_17 (MakeStruct (n_def (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_agg16))))))) (sum (ApplyAggOp Sum () ( (Ref __uid_agg16)))) (sumsq (ApplyAggOp Sum () ( (Apply `**` Float64 (Ref __uid_agg16) (ApplyIR toFloat64 Float64 (I32 2)))))) (min (ApplyAggOp Min () ( (Ref __uid_agg16)))) (max (ApplyAggOp Max () ( (Ref __uid_agg16))))) (Let __cse_6 (GetField sum (Ref __uid_17)) (Let __cse_7 (GetField n_def (Ref __uid_17)) (Let __uid_18 (ApplyBinaryPrimOp FloatingPointDivide (Ref __cse_6) (ApplyIR toFloat64 Float64 (Ref __cse_7))) (MakeStruct (mean (Ref __uid_18)) (stdev (Apply sqrt Float64 (ApplyBinaryPrimOp FloatingPointDivide (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Subtract (GetField sumsq (Ref __uid_17)) (ApplyBinaryPrimOp Multiply (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (I32 2)) (Ref __uid_18)) (Ref __cse_6))) (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (Ref __cse_7)) (Apply `**` Float64 (Ref __uid_18) (ApplyIR toFloat64 Float64 (I32 2))))) (ApplyIR toFloat64 Float64 (Ref __cse_7))))) (min (GetField min (Ref __uid_17))) (max (GetField max (Ref __uid_17))) (n (Ref __cse_7)) (sum (Ref __cse_6)))))))))) (gq_stats (SelectFields (mean stdev min max) (AggLet __uid_agg19 False (ApplyIR toFloat64 Float64 (GetField GQ (Ref g))) (Let __uid_20 (MakeStruct (n_def (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_agg19))))))) (sum (ApplyAggOp Sum () ( (Ref __uid_agg19)))) (sumsq (ApplyAggOp Sum () ( (Apply `**` Float64 (Ref __uid_agg19) (ApplyIR toFloat64 Float64 (I32 2)))))) (min (ApplyAggOp Min () ( (Ref __uid_agg19)))) (max (ApplyAggOp Max () ( (Ref __uid_agg19))))) (Let __cse_8 (GetField sum (Ref __uid_20)) (Let __cse_9 (GetField n_def (Ref __uid_20)) (Let __uid_21 (ApplyBinaryPrimOp FloatingPointDivide (Ref __cse_8) (ApplyIR toFloat64 Float64 (Ref __cse_9))) (MakeStruct (mean (Ref __uid_21)) (stdev (Apply sqrt Float64 (ApplyBinaryPrimOp FloatingPointDivide (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Subtract (GetField sumsq (Ref __uid_20)) (ApplyBinaryPrimOp Multiply (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (I32 2)) (Ref __uid_21)) (Ref __cse_8))) (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (Ref __cse_9)) (Apply `**` Float64 (Ref __uid_21) (ApplyIR toFloat64 Float64 (I32 2))))) (ApplyIR toFloat64 Float64 (Ref __cse_9))))) (min (GetField min (Ref __uid_20))) (max (GetField max (Ref __uid_20))) (n (Ref __cse_9)) (sum (Ref __cse_8)))))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (Ref __cse_10)) (ApplyIR toFloat64 Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (Ref __cse_10) (Ref __cse_11)) (Ref __cse_12))))) (n_called (Ref __cse_10)) (n_not_called (Ref __cse_11)) (n_filtered (Ref __cse_12)) (n_hom_ref (Ref __cse_13)) (n_het (Ref __cse_14)) (n_hom_var (ApplyBinaryPrimOp Subtract (ApplyBinaryPrimOp Subtract (Ref __cse_10) (Ref __cse_13)) (Ref __cse_14))) (n_non_ref (ApplyBinaryPrimOp Subtract (Ref __cse_10) (Ref __cse_13))) (n_singleton (GetField n_singleton (Ref __uid_27))) (n_snp (ApplyBinaryPrimOp Add (ApplyIR get Int64 (Ref __cse_15) (I32 8) (Ref __cse_16)) (ApplyIR get Int64 (Ref __cse_15) (I32 9) (Ref __cse_16)))) (n_insertion (ApplyIR get Int64 (Ref __cse_15) (I32 3) (Ref __cse_16))) (n_deletion (ApplyIR get Int64 (Ref __cse_15) (I32 4) (Ref __cse_16))) (n_transition (ApplyIR get Int64 (Ref __cse_15) (I32 8) (Ref __cse_16))) (n_transversion (ApplyIR get Int64 (Ref __cse_15) (I32 9) (Ref __cse_16))) (n_star (ApplyIR get Int64 (Ref __cse_15) (I32 6) (Ref __cse_16)))) (InsertFields (Ref __uid_28) None (r_ti_tv (If (ApplyComparisonOp NEQ (GetField n_transversion (Ref __uid_28)) (ApplyIR toInt64 Int64 (I32 0))) (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (GetField n_transition (Ref __uid_28))) (ApplyIR toFloat64 Float64 (GetField n_transversion (Ref __uid_28)))) (NA Float64))) (r_het_hom_var (If (ApplyComparisonOp NEQ (GetField n_hom_var (Ref __uid_28)) (ApplyIR toInt64 Int64 (I32 0))) (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (GetField n_het (Ref __uid_28))) (ApplyIR toFloat64 Float64 (GetField n_hom_var (Ref __uid_28)))) (NA Float64))) (r_insertion_deletion (If (ApplyComparisonOp NEQ (GetField n_deletion (Ref __uid_28)) (ApplyIR toInt64 Int64 (I32 0))) (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (GetField n_insertion (Ref __uid_28))) (ApplyIR toFloat64 Float64 (GetField n_deletion (Ref __uid_28)))) (NA Float64))))))))))))))))) (SelectFields (locus alleles rsid qual filters info) (MakeStruct (locus (GetField locus (Ref va))) (alleles (GetField alleles (Ref va))) (rsid (GetField rsid (Ref va))) (qual (GetField qual (Ref va))) (filters (GetField filters (Ref va))) (info (GetField info (Ref va))) (__uid_4 (GetField __uid_4 (Ref va))) (__uid_5 (GetField __uid_5 (Ref va)))))) (Let __cse_17 (GetField sample_qc (Ref sa)) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (Ref __cse_17))) (ApplyIR toFloat64 Float64 (I32 4))) (ApplyComparisonOp GTEQ (GetField call_rate (Ref __cse_17)) (F64 0.97))) (False)))) (Let __uid_30 (ApplyAggOp Count () ()) (If (ApplyComparisonOp EQ (Ref __uid_30) (ApplyIR toInt64 Int64 (I32 0))) (NA Float64) (AggLet __cse_19 False (GetField GT (Ref g)) (AggLet __cse_18 False (GetField AD (Ref g)) (AggLet __cse_20 False (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR indexArray Int32 (Ref __cse_18) (I32 1)) (ApplyIR sum Int32 (ArrayFilter __uid_29 (Ref __cse_18) (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_29)))))) (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (AggFilter False (ApplyUnaryPrimOp Bang (ApplySpecial `||` Boolean (ApplySpecial `||` Boolean (ApplySpecial `&&` Boolean (Apply isHomRef Boolean (Ref __cse_19)) (ApplyComparisonOp LTEQ (ApplyIR toFloat64 Float64 (Ref __cse_20)) (F64 0.1))) (ApplySpecial `&&` Boolean (ApplySpecial `&&` Boolean (Apply isHet Boolean (Ref __cse_19)) (ApplyComparisonOp GTEQ (ApplyIR toFloat64 Float64 (Ref __cse_20)) (F64 0.25))) (ApplyComparisonOp LTEQ (ApplyIR toFloat64 Float64 (Ref __cse_20)) (F64 0.75)))) (ApplySpecial `&&` Boolean (Apply isHomVar Boolean (Ref __cse_19)) (ApplyComparisonOp GTEQ (ApplyIR toFloat64 Float64 (Ref __cse_20)) (F64 0.9))))) (ApplyAggOp Count () ()))) (ApplyIR toFloat64 Float64 (Ref __uid_30))))))))) 2020-01-07 21:47:47 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 3.002ms, total 9.382ms 2020-01-07 21:47:47 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 15.192ms, total 24.718ms 2020-01-07 21:47:47 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 19.547ms, total 44.393ms 2020-01-07 21:47:47 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 38.314ms, total 82.992ms 2020-01-07 21:47:47 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 6.207ms, total 89.530ms 2020-01-07 21:47:47 root: INFO: Prune: InsertFields: eliminating field 'pheno' 2020-01-07 21:47:47 root: INFO: Prune: InsertFields: eliminating field '__uid_4' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_singleton' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'sumsq' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'min' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'max' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'stdev' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'min' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'max' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'sum' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'gq_stats' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_called' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_not_called' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_filtered' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_hom_ref' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_het' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_hom_var' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_non_ref' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_singleton' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_snp' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_insertion' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_deletion' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_transition' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_transversion' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_star' 2020-01-07 21:47:48 root: INFO: Prune: InsertFields: eliminating field 'r_ti_tv' 2020-01-07 21:47:48 root: INFO: Prune: InsertFields: eliminating field 'r_het_hom_var' 2020-01-07 21:47:48 root: INFO: Prune: InsertFields: eliminating field 'r_insertion_deletion' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'rsid' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'qual' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'filters' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'info' 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 27.433ms, total 117.110ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 1.179ms, total 118.401ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 6.339ms, total 124.799ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 3.579ms, total 128.475ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 18.602ms, total 147.164ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.773ms, total 148.065ms 2020-01-07 21:47:48 root: INFO: Prune: InsertFields: eliminating field '__uid_5' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_hom_ref' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'n_het' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'allele_type_counts' 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 5.100ms, total 153.216ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.661ms, total 154.024ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.601ms, total 154.674ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 2.648ms, total 157.381ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 4.007ms, total 161.458ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.515ms, total 162.053ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 3.899ms, total 166.002ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 159.705ms, total 166.079ms, tagged coverage 98.7 2020-01-07 21:47:48 root: INFO: optimize Lowerer, initial IR: after: IR size 147: (MatrixAggregate (MatrixFilterCols (MatrixMapRows (MatrixMapCols () (MatrixRead Matrix{global:Struct{},col_key:[s],col:Struct{s:String},row_key:[[locus,alleles]],row:Struct{locus:Locus(GRCh37),alleles:Array[String]},entry:Struct{GT:Call,AD:Array[+Int32],DP:Int32}} False False "{\"name\":\"MatrixNativeReader\",\"path\":\"data/1kg.mt\",\"_spec\":null}") (AggLet __iruid_3002 False (GetField GT (Ref g)) (InsertFields (SelectFields () (Ref sa)) None (sample_qc (Let __iruid_3003 (MakeStruct (n_called (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_3002))))))) (n_not_called (ApplyAggOp Sum () ( (Apply toInt64 Int64 (IsNA (Ref __iruid_3002)))))) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (ApplyAggOp Count () ())))) (Let __iruid_3004 (GetField n_called (Ref __iruid_3003)) (Let __iruid_3005 (MakeStruct (dp_stats (AggLet __iruid_3006 False (Cast Float64 (GetField DP (Ref g))) (Let __iruid_3007 (MakeStruct (n_def (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_3006))))))) (sum (ApplyAggOp Sum () ( (Ref __iruid_3006))))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_3007)) (Cast Float64 (GetField n_def (Ref __iruid_3007))))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (Ref __iruid_3004)) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (Ref __iruid_3004) (GetField n_not_called (Ref __iruid_3003))) (GetField n_filtered (Ref __iruid_3003))))))) (Ref __iruid_3005)))))))) (MakeStruct (locus (GetField locus (Ref va))) (alleles (GetField alleles (Ref va))))) (Let __iruid_3008 (GetField sample_qc (Ref sa)) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (Ref __iruid_3008))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (Ref __iruid_3008)) (F64 0.97))) (False)))) (Let __iruid_3009 (ApplyAggOp Count () ()) (If (ApplyComparisonOp EQ (Ref __iruid_3009) (I64 0)) (NA Float64) (AggLet __iruid_3010 False (GetField GT (Ref g)) (AggLet __iruid_3011 False (GetField AD (Ref g)) (AggLet __iruid_3012 False (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (Ref __iruid_3011) (I32 1)) (ArrayFold __iruid_3013 __iruid_3014 (ArrayFilter __iruid_3015 (Ref __iruid_3011) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_3015)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_3013) (Ref __iruid_3014)))) (ApplyBinaryPrimOp FloatingPointDivide (Let __iruid_3016 (AggFilter False (ApplyUnaryPrimOp Bang (ApplySpecial `||` Boolean (ApplySpecial `||` Boolean (ApplySpecial `&&` Boolean (Apply isHomRef Boolean (Ref __iruid_3010)) (ApplyComparisonOp LTEQ (Cast Float64 (Ref __iruid_3012)) (F64 0.1))) (ApplySpecial `&&` Boolean (ApplySpecial `&&` Boolean (Apply isHet Boolean (Ref __iruid_3010)) (ApplyComparisonOp GTEQ (Cast Float64 (Ref __iruid_3012)) (F64 0.25))) (ApplyComparisonOp LTEQ (Cast Float64 (Ref __iruid_3012)) (F64 0.75)))) (ApplySpecial `&&` Boolean (Apply isHomVar Boolean (Ref __iruid_3010)) (ApplyComparisonOp GTEQ (Cast Float64 (Ref __iruid_3012)) (F64 0.9))))) (ApplyAggOp Count () ())) (Cast Float64 (Ref __iruid_3016))) (Cast Float64 (Ref __iruid_3009))))))))) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- Verify : 0.088ms, total 168.425ms 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- LoweringTransformation : 14.726ms, total 183.215ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- Verify : 0.497ms, total 183.814ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable : 15.526ms, total 183.860ms, tagged coverage 98.6 2020-01-07 21:47:48 root: INFO: optimize LowerMatrixToTable, post Lowering: before: IR size 315: (RelationalLet __iruid_3025 (TableAggregate (TableMapGlobals (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32}]}} False "{\"name\":\"TableNativeZippedReader\",\"pathLeft\":\"data/1kg.mt/rows\",\"pathRight\":\"data/1kg.mt/entries\",\"_specLeft\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AC:Array[Int32],AF:Array[Float64],AN:Int32,BaseQRankSum:Float64,ClippingRankSum:Float64,DP:Int32,DS:Boolean,FS:Float64,HaplotypeScore:Float64,InbreedingCoeff:Float64,MLEAC:Array[Int32],MLEAF:Array[Float64],MQ:Float64,MQ0:Int32,MQRankSum:Float64,QD:Float64,ReadPosRankSum:Float64,set:String}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}},\"_specRight\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[],row:Struct{`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32,GQ:Int32,PL:Array[+Int32]}]}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}}}") (InsertFields (Ref global) None (__cols (GetField rows (TableCollect (TableRead Table{global:Struct{},key:[],row:Struct{s:String}} False "{\"name\":\"TableNativeReader\",\"path\":\"data/1kg.mt/cols\",\"_spec\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[s],row:Struct{s:String}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[284]}}}}")))))) (AggLet va False (SelectFields (locus alleles) (Ref row)) (AggArrayPerElement __element_idx __result_idx False True (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let sa (ArrayRef (GetField __cols (Ref global)) (Ref __result_idx)) (AggLet sa False (ArrayRef (GetField __cols (Ref global)) (Ref __element_idx)) (AggLet g False (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __element_idx)) (AggFilter False (ApplyUnaryPrimOp Bang (IsNA (Ref g))) (MakeStruct (__iruid_3023 (AggLet __iruid_3002 False (GetField GT (Ref g)) (MakeStruct (__iruid_3017 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_3002))))))) (__iruid_3018 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (IsNA (Ref __iruid_3002)))))) (__iruid_3019 (ApplyAggOp Count () ())) (__iruid_3022 (AggLet __iruid_3006 False (Cast Float64 (GetField DP (Ref g))) (MakeStruct (__iruid_3020 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_3006))))))) (__iruid_3021 (ApplyAggOp Sum () ( (Ref __iruid_3006))))))))))))))) (ArrayLen (GetField __cols (Ref global)))))) (TableAggregate (TableMapGlobals (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableMapGlobals (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32}]}} False "{\"name\":\"TableNativeZippedReader\",\"pathLeft\":\"data/1kg.mt/rows\",\"pathRight\":\"data/1kg.mt/entries\",\"_specLeft\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AC:Array[Int32],AF:Array[Float64],AN:Int32,BaseQRankSum:Float64,ClippingRankSum:Float64,DP:Int32,DS:Boolean,FS:Float64,HaplotypeScore:Float64,InbreedingCoeff:Float64,MLEAC:Array[Int32],MLEAF:Array[Float64],MQ:Float64,MQ0:Int32,MQRankSum:Float64,QD:Float64,ReadPosRankSum:Float64,set:String}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}},\"_specRight\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[],row:Struct{`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32,GQ:Int32,PL:Array[+Int32]}]}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}}}") (InsertFields (Ref global) None (__cols (GetField rows (TableCollect (TableRead Table{global:Struct{},key:[],row:Struct{s:String}} False "{\"name\":\"TableNativeReader\",\"path\":\"data/1kg.mt/cols\",\"_spec\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[s],row:Struct{s:String}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[284]}}}}")))))) (InsertFields (Ref global) None (__cols (Let __iruid_3026 (RelationalRef __iruid_3025 Array[Struct{__iruid_3023:Struct{__iruid_3017:Int64,__iruid_3018:Int64,__iruid_3019:Int64,__iruid_3022:Struct{__iruid_3020:Int64,__iruid_3021:Float64}}}]) (ArrayMap __iruid_3024 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __cols_array (GetField __cols (Ref global)) (Let sa (ArrayRef (Ref __cols_array) (Ref __iruid_3024)) (Let __iruid_3026 (RelationalRef __iruid_3025 Array[Struct{__iruid_3023:Struct{__iruid_3017:Int64,__iruid_3018:Int64,__iruid_3019:Int64,__iruid_3022:Struct{__iruid_3020:Int64,__iruid_3021:Float64}}}]) (Let __iruid_3027 (ArrayRef (Ref __iruid_3026) (Ref __iruid_3024)) (Let __iruid_3023 (GetField __iruid_3023 (Ref __iruid_3027)) (Let __iruid_3022 (GetField __iruid_3022 (Ref __iruid_3023)) (Let __iruid_3019 (GetField __iruid_3019 (Ref __iruid_3023)) (Let __iruid_3018 (GetField __iruid_3018 (Ref __iruid_3023)) (Let __iruid_3017 (GetField __iruid_3017 (Ref __iruid_3023)) (InsertFields (SelectFields () (Ref sa)) None (sample_qc (Let __iruid_3003 (MakeStruct (n_called (Ref __iruid_3017)) (n_not_called (Ref __iruid_3018)) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (Ref __iruid_3019)))) (Let __iruid_3004 (GetField n_called (Ref __iruid_3003)) (Let __iruid_3005 (MakeStruct (dp_stats (Let __iruid_3021 (GetField __iruid_3021 (Ref __iruid_3022)) (Let __iruid_3020 (GetField __iruid_3020 (Ref __iruid_3022)) (Let __iruid_3007 (MakeStruct (n_def (Ref __iruid_3020)) (sum (Ref __iruid_3021))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_3007)) (Cast Float64 (GetField n_def (Ref __iruid_3007)))))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (Ref __iruid_3004)) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (Ref __iruid_3004) (GetField n_not_called (Ref __iruid_3003))) (GetField n_filtered (Ref __iruid_3003))))))) (Ref __iruid_3005)))))))))))))))))))) (InsertFields (Let __iruid_3028 (MakeStruct) (MakeStruct (locus (GetField locus (SelectFields (locus alleles) (Ref row)))) (alleles (GetField alleles (SelectFields (locus alleles) (Ref row)))))) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row))))) (InsertFields (Ref global) None (newColIdx (ArrayFilter i (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let sa (ArrayRef (GetField __cols (Ref global)) (Ref i)) (Let __iruid_3008 (GetField sample_qc (Ref sa)) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (Ref __iruid_3008))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (Ref __iruid_3008)) (F64 0.97))) (False)))))))) (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap i (GetField newColIdx (Ref global)) (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref i)))))) (SelectFields (__cols) (InsertFields (Ref global) None (__cols (ArrayMap i (GetField newColIdx (Ref global)) (ArrayRef (GetField __cols (Ref global)) (Ref i))))))) (AggLet __entries_field False (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (AggLet __cols_field False (GetField __cols (Ref global)) (AggExplode __iruid_3029 False (ArrayFilter __iruid_3029 (ArrayRange (I32 0) (ArrayLen (Ref __entries_field)) (I32 1)) (ApplyUnaryPrimOp Bang (IsNA (ArrayRef (Ref __entries_field) (Ref __iruid_3029))))) (AggLet sa False (ArrayRef (Ref __cols_field) (Ref __iruid_3029)) (AggLet g False (ArrayRef (Ref __entries_field) (Ref __iruid_3029)) (Let __iruid_3009 (ApplyAggOp Count () ()) (If (ApplyComparisonOp EQ (Ref __iruid_3009) (I64 0)) (NA Float64) (AggLet __iruid_3010 False (GetField GT (Ref g)) (AggLet __iruid_3011 False (GetField AD (Ref g)) (AggLet __iruid_3012 False (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (Ref __iruid_3011) (I32 1)) (ArrayFold __iruid_3013 __iruid_3014 (ArrayFilter __iruid_3015 (Ref __iruid_3011) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_3015)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_3013) (Ref __iruid_3014)))) (ApplyBinaryPrimOp FloatingPointDivide (Let __iruid_3016 (AggFilter False (ApplyUnaryPrimOp Bang (ApplySpecial `||` Boolean (ApplySpecial `||` Boolean (ApplySpecial `&&` Boolean (Apply isHomRef Boolean (Ref __iruid_3010)) (ApplyComparisonOp LTEQ (Cast Float64 (Ref __iruid_3012)) (F64 0.1))) (ApplySpecial `&&` Boolean (ApplySpecial `&&` Boolean (Apply isHet Boolean (Ref __iruid_3010)) (ApplyComparisonOp GTEQ (Cast Float64 (Ref __iruid_3012)) (F64 0.25))) (ApplyComparisonOp LTEQ (Cast Float64 (Ref __iruid_3012)) (F64 0.75)))) (ApplySpecial `&&` Boolean (Apply isHomVar Boolean (Ref __iruid_3010)) (ApplyComparisonOp GTEQ (Cast Float64 (Ref __iruid_3012)) (F64 0.9))))) (ApplyAggOp Count () ())) (Cast Float64 (Ref __iruid_3016))) (Cast Float64 (Ref __iruid_3009))))))))))))))) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 1.041ms, total 189.182ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 1.118ms, total 190.388ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 8.030ms, total 198.477ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 8.845ms, total 207.415ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.927ms, total 208.454ms 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'locus' 2020-01-07 21:47:48 root: INFO: Prune: MakeStruct: eliminating field 'alleles' 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 8.699ms, total 217.216ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 1.164ms, total 218.465ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 1.010ms, total 219.531ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 2.642ms, total 222.250ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 6.215ms, total 228.538ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.730ms, total 229.368ms 2020-01-07 21:47:48 root: INFO: Prune: InsertFields: eliminating field '__cols' 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 6.965ms, total 236.393ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.973ms, total 237.479ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.946ms, total 238.481ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 1.609ms, total 240.142ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 4.287ms, total 244.476ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.778ms, total 245.311ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 6.639ms, total 251.999ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 63.919ms, total 252.055ms, tagged coverage 98.0 2020-01-07 21:47:48 root: INFO: optimize LowerMatrixToTable, post Lowering: after: IR size 252: (RelationalLet __iruid_3025 (TableAggregate (TableKeyBy () False (TableMapGlobals (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,DP:Int32}]}} False "{\"name\":\"TableNativeZippedReader\",\"pathLeft\":\"data/1kg.mt/rows\",\"pathRight\":\"data/1kg.mt/entries\",\"_specLeft\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AC:Array[Int32],AF:Array[Float64],AN:Int32,BaseQRankSum:Float64,ClippingRankSum:Float64,DP:Int32,DS:Boolean,FS:Float64,HaplotypeScore:Float64,InbreedingCoeff:Float64,MLEAC:Array[Int32],MLEAF:Array[Float64],MQ:Float64,MQ0:Int32,MQRankSum:Float64,QD:Float64,ReadPosRankSum:Float64,set:String}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}},\"_specRight\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[],row:Struct{`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32,GQ:Int32,PL:Array[+Int32]}]}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}}}") (InsertFields (Ref global) None (__cols (GetField rows (TableCollect (TableRead Table{global:Struct{},key:[],row:Struct{}} False "{\"name\":\"TableNativeReader\",\"path\":\"data/1kg.mt/cols\",\"_spec\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[s],row:Struct{s:String}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[284]}}}}"))))))) (AggArrayPerElement __iruid_3115 __iruid_3116 False True (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (AggLet __iruid_3117 False (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_3115)) (AggFilter False (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_3117))) (MakeStruct (__iruid_3023 (AggLet __iruid_3118 False (GetField GT (Ref __iruid_3117)) (MakeStruct (__iruid_3017 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_3118))))))) (__iruid_3018 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (IsNA (Ref __iruid_3118)))))) (__iruid_3019 (ApplyAggOp Count () ())) (__iruid_3022 (AggLet __iruid_3119 False (Cast Float64 (GetField DP (Ref __iruid_3117))) (MakeStruct (__iruid_3020 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_3119))))))) (__iruid_3021 (ApplyAggOp Sum () ( (Ref __iruid_3119))))))))))))) (ArrayLen (GetField __cols (Ref global))))) (TableAggregate (TableMapGlobals (TableMapRows (TableMapGlobals (TableMapRows (TableKeyBy () False (TableMapGlobals (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32]}]}} False "{\"name\":\"TableNativeZippedReader\",\"pathLeft\":\"data/1kg.mt/rows\",\"pathRight\":\"data/1kg.mt/entries\",\"_specLeft\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AC:Array[Int32],AF:Array[Float64],AN:Int32,BaseQRankSum:Float64,ClippingRankSum:Float64,DP:Int32,DS:Boolean,FS:Float64,HaplotypeScore:Float64,InbreedingCoeff:Float64,MLEAC:Array[Int32],MLEAF:Array[Float64],MQ:Float64,MQ0:Int32,MQRankSum:Float64,QD:Float64,ReadPosRankSum:Float64,set:String}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}},\"_specRight\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[],row:Struct{`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32,GQ:Int32,PL:Array[+Int32]}]}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}}}") (Let __iruid_3120 (GetField rows (TableCollect (TableRead Table{global:Struct{},key:[],row:Struct{}} False "{\"name\":\"TableNativeReader\",\"path\":\"data/1kg.mt/cols\",\"_spec\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[s],row:Struct{s:String}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[284]}}}}"))) (InsertFields (Ref global) ("__cols") (__cols (ArrayMap __iruid_3121 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_3120)) (I32 1)) (Let __iruid_3122 (GetField __iruid_3023 (ArrayRef (RelationalRef __iruid_3025 Array[Struct{__iruid_3023:Struct{__iruid_3017:Int64,__iruid_3018:Int64,__iruid_3019:Int64,__iruid_3022:Struct{__iruid_3020:Int64,__iruid_3021:Float64}}}]) (Ref __iruid_3121))) (Let __iruid_3123 (GetField __iruid_3022 (Ref __iruid_3122)) (InsertFields (ArrayRef (Ref __iruid_3120) (Ref __iruid_3121)) None (sample_qc (Let __iruid_3124 (MakeStruct (n_called (GetField __iruid_3017 (Ref __iruid_3122))) (n_not_called (GetField __iruid_3018 (Ref __iruid_3122))) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (GetField __iruid_3019 (Ref __iruid_3122))))) (Let __iruid_3125 (GetField n_called (Ref __iruid_3124)) (MakeStruct (dp_stats (Let __iruid_3126 (MakeStruct (n_def (GetField __iruid_3020 (Ref __iruid_3123))) (sum (GetField __iruid_3021 (Ref __iruid_3123)))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_3126)) (Cast Float64 (GetField n_def (Ref __iruid_3126)))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (Ref __iruid_3125)) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (Ref __iruid_3125) (GetField n_not_called (Ref __iruid_3124))) (GetField n_filtered (Ref __iruid_3124))))))))))))))))))) (InsertFields (Literal Struct{} ) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row))))) (InsertFields (Ref global) None (newColIdx (ArrayFilter __iruid_3127 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_3128 (GetField sample_qc (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_3127))) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (Ref __iruid_3128))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (Ref __iruid_3128)) (F64 0.97))) (False))))))) (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap __iruid_3129 (GetField newColIdx (Ref global)) (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_3129)))))) (SelectFields () (Ref global))) (AggLet __iruid_3130 False (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (AggExplode __iruid_3131 False (ArrayFilter __iruid_3132 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_3130)) (I32 1)) (ApplyUnaryPrimOp Bang (IsNA (ArrayRef (Ref __iruid_3130) (Ref __iruid_3132))))) (AggLet __iruid_3133 False (ArrayRef (Ref __iruid_3130) (Ref __iruid_3131)) (Let __iruid_3134 (ApplyAggOp Count () ()) (If (ApplyComparisonOp EQ (Ref __iruid_3134) (I64 0)) (NA Float64) (AggLet __iruid_3135 False (GetField GT (Ref __iruid_3133)) (AggLet __iruid_3136 False (GetField AD (Ref __iruid_3133)) (AggLet __iruid_3137 False (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (Ref __iruid_3136) (I32 1)) (ArrayFold __iruid_3138 __iruid_3139 (ArrayFilter __iruid_3140 (Ref __iruid_3136) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_3140)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_3138) (Ref __iruid_3139)))) (ApplyBinaryPrimOp FloatingPointDivide (Let __iruid_3141 (AggFilter False (ApplyUnaryPrimOp Bang (ApplySpecial `||` Boolean (ApplySpecial `||` Boolean (ApplySpecial `&&` Boolean (Apply isHomRef Boolean (Ref __iruid_3135)) (ApplyComparisonOp LTEQ (Cast Float64 (Ref __iruid_3137)) (F64 0.1))) (ApplySpecial `&&` Boolean (ApplySpecial `&&` Boolean (Apply isHet Boolean (Ref __iruid_3135)) (ApplyComparisonOp GTEQ (Cast Float64 (Ref __iruid_3137)) (F64 0.25))) (ApplyComparisonOp LTEQ (Cast Float64 (Ref __iruid_3137)) (F64 0.75)))) (ApplySpecial `&&` Boolean (Apply isHomVar Boolean (Ref __iruid_3135)) (ApplyComparisonOp GTEQ (Cast Float64 (Ref __iruid_3137)) (F64 0.9))))) (ApplyAggOp Count () ())) (Cast Float64 (Ref __iruid_3141))) (Cast Float64 (Ref __iruid_3134))))))))))))) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- Verify : 0.085ms, total 256.360ms 2020-01-07 21:47:48 root: INFO: interpreting non compilable node: TableCollect 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 root: INFO: encoder cache hit 2020-01-07 21:47:48 SparkContext: INFO: Starting job: collect at RVD.scala:676 2020-01-07 21:47:48 DAGScheduler: INFO: Got job 8 (collect at RVD.scala:676) with 1 output partitions 2020-01-07 21:47:48 DAGScheduler: INFO: Final stage: ResultStage 8 (collect at RVD.scala:676) 2020-01-07 21:47:48 DAGScheduler: INFO: Parents of final stage: List() 2020-01-07 21:47:48 DAGScheduler: INFO: Missing parents: List() 2020-01-07 21:47:48 DAGScheduler: INFO: Submitting ResultStage 8 (MapPartitionsRDD[44] at mapPartitions at ContextRDD.scala:217), which has no missing parents 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_18 stored as values in memory (estimated size 10.3 KB, free 365.6 MB) 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_18_piece0 stored as bytes in memory (estimated size 5.5 KB, free 365.6 MB) 2020-01-07 21:47:48 BlockManagerInfo: INFO: Added broadcast_18_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 5.5 KB, free: 366.2 MB) 2020-01-07 21:47:48 SparkContext: INFO: Created broadcast 18 from broadcast at DAGScheduler.scala:1161 2020-01-07 21:47:48 DAGScheduler: INFO: Submitting 1 missing tasks from ResultStage 8 (MapPartitionsRDD[44] at mapPartitions at ContextRDD.scala:217) (first 15 tasks are for partitions Vector(0)) 2020-01-07 21:47:48 TaskSchedulerImpl: INFO: Adding task set 8.0 with 1 tasks 2020-01-07 21:47:48 TaskSetManager: INFO: Starting task 0.0 in stage 8.0 (TID 43, localhost, executor driver, partition 0, PROCESS_LOCAL, 7653 bytes) 2020-01-07 21:47:48 Executor: INFO: Running task 0.0 in stage 8.0 (TID 43) 2020-01-07 21:47:48 Executor: INFO: Finished task 0.0 in stage 8.0 (TID 43). 3897 bytes result sent to driver 2020-01-07 21:47:48 TaskSetManager: INFO: Finished task 0.0 in stage 8.0 (TID 43) in 9 ms on localhost (executor driver) (1/1) 2020-01-07 21:47:48 TaskSchedulerImpl: INFO: Removed TaskSet 8.0, whose tasks have all completed, from pool 2020-01-07 21:47:48 DAGScheduler: INFO: ResultStage 8 (collect at RVD.scala:676) finished in 0.017 s 2020-01-07 21:47:48 DAGScheduler: INFO: Job 8 finished: collect at RVD.scala:676, took 0.019925 s 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 root: INFO: took 43.325ms 2020-01-07 21:47:48 root: INFO: interpreting non compilable node: TableAggregate 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 root: INFO: encoder cache hit 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_19 stored as values in memory (estimated size 4.6 KB, free 365.5 MB) 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_19_piece0 stored as bytes in memory (estimated size 2.1 KB, free 365.5 MB) 2020-01-07 21:47:48 BlockManagerInfo: INFO: Added broadcast_19_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 2.1 KB, free: 366.2 MB) 2020-01-07 21:47:48 SparkContext: INFO: Created broadcast 19 from broadcast at SparkBackend.scala:21 2020-01-07 21:47:48 root: INFO: optimize Compile: before: IR size 29: (Let __iruid_3142 (ResultOp2 0 ( (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))))) (MakeTuple (0) (Let __iruid_3146 (GetTupleElement 0 (Ref __iruid_3142)) (ArrayMap __iruid_3116 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_3146)) (I32 1)) (Let __iruid_3143 (ArrayRef (Ref __iruid_3146) (Ref __iruid_3116)) (MakeStruct (__iruid_3023 (MakeStruct (__iruid_3017 (GetTupleElement 0 (Ref __iruid_3143))) (__iruid_3018 (GetTupleElement 1 (Ref __iruid_3143))) (__iruid_3019 (GetTupleElement 2 (Ref __iruid_3143))) (__iruid_3022 (MakeStruct (__iruid_3020 (GetTupleElement 3 (Ref __iruid_3143))) (__iruid_3021 (GetTupleElement 4 (Ref __iruid_3143))))))))))))) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.627ms, total 331.961ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.108ms, total 332.138ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.183ms, total 332.364ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.689ms, total 333.091ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.174ms, total 333.324ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.937ms, total 334.316ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.242ms, total 334.613ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.091ms, total 334.752ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.171ms, total 334.962ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.636ms, total 335.638ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.093ms, total 335.779ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.825ms, total 336.642ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.178ms, total 336.877ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.078ms, total 337.003ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.161ms, total 337.200ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.624ms, total 337.858ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.099ms, total 338.009ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.743ms, total 338.789ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 7.517ms, total 338.847ms, tagged coverage 88.6 2020-01-07 21:47:48 root: INFO: optimize Compile: after: IR size 29: (Let __iruid_3155 (ResultOp2 0 ( (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))))) (MakeTuple (0) (Let __iruid_3156 (GetTupleElement 0 (Ref __iruid_3155)) (ArrayMap __iruid_3157 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_3156)) (I32 1)) (Let __iruid_3158 (ArrayRef (Ref __iruid_3156) (Ref __iruid_3157)) (MakeStruct (__iruid_3023 (MakeStruct (__iruid_3017 (GetTupleElement 0 (Ref __iruid_3158))) (__iruid_3018 (GetTupleElement 1 (Ref __iruid_3158))) (__iruid_3019 (GetTupleElement 2 (Ref __iruid_3158))) (__iruid_3022 (MakeStruct (__iruid_3020 (GetTupleElement 3 (Ref __iruid_3158))) (__iruid_3021 (GetTupleElement 4 (Ref __iruid_3158))))))))))))) 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C34. 2020-01-07 21:47:48 root: INFO: instruction count: 84: is/hail/codegen/generated/C34.apply 2020-01-07 21:47:48 root: INFO: instruction count: 16: is/hail/codegen/generated/C34.apply 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C34.newAggState 2020-01-07 21:47:48 root: INFO: instruction count: 17: is/hail/codegen/generated/C34.setAggState 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C34.getAggOffset 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C34.setNumSerialized 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C34.setSerializedAgg 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C34.getSerializedAgg 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C34.m7_create_states_group0 2020-01-07 21:47:48 root: INFO: instruction count: 19: is/hail/codegen/generated/C34.m8_create_states_group0 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C34.m9_new_state_group0 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C34.m10_create_states_group0 2020-01-07 21:47:48 root: INFO: instruction count: 19: is/hail/codegen/generated/C34.m11_create_states_group0 2020-01-07 21:47:48 root: INFO: instruction count: 53: is/hail/codegen/generated/C34.m12_load_group0 2020-01-07 21:47:48 root: INFO: instruction count: 36: is/hail/codegen/generated/C34.m13_store_group0 2020-01-07 21:47:48 root: INFO: instruction count: 113: is/hail/codegen/generated/C34.m14_load_group0 2020-01-07 21:47:48 root: INFO: instruction count: 57: is/hail/codegen/generated/C34.m15_array_nested_result_group0 2020-01-07 21:47:48 root: INFO: instruction count: 137: is/hail/codegen/generated/C34.m16_aggs_result_group0 2020-01-07 21:47:48 root: INFO: instruction count: 36: is/hail/codegen/generated/C34.m17_store_group0 2020-01-07 21:47:48 root: INFO: instruction count: 486: is/hail/codegen/generated/C34.m18_method 2020-01-07 21:47:48 root: INFO: instruction count: 46: is/hail/codegen/generated/C34.m19_method 2020-01-07 21:47:48 root: INFO: instruction count: 275: is/hail/codegen/generated/C34.m20_method 2020-01-07 21:47:48 root: INFO: instruction count: 14: is/hail/codegen/generated/C34.setPartitionIndex 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C34.addPartitionRegion 2020-01-07 21:47:48 root: INFO: Aggregate: useTreeAggregate=true 2020-01-07 21:47:48 root: INFO: Aggregate: commutative=true 2020-01-07 21:47:48 SparkContext: INFO: Starting job: runJob at RVD.scala:605 2020-01-07 21:47:48 DAGScheduler: INFO: Got job 9 (runJob at RVD.scala:605) with 2 output partitions 2020-01-07 21:47:48 DAGScheduler: INFO: Final stage: ResultStage 9 (runJob at RVD.scala:605) 2020-01-07 21:47:48 DAGScheduler: INFO: Parents of final stage: List() 2020-01-07 21:47:48 DAGScheduler: INFO: Missing parents: List() 2020-01-07 21:47:48 DAGScheduler: INFO: Submitting ResultStage 9 (MapPartitionsRDD[49] at mapPartitions at ContextRDD.scala:217), which has no missing parents 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_20 stored as values in memory (estimated size 71.8 KB, free 365.5 MB) 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_20_piece0 stored as bytes in memory (estimated size 33.8 KB, free 365.4 MB) 2020-01-07 21:47:48 BlockManagerInfo: INFO: Added broadcast_20_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 33.8 KB, free: 366.2 MB) 2020-01-07 21:47:48 SparkContext: INFO: Created broadcast 20 from broadcast at DAGScheduler.scala:1161 2020-01-07 21:47:48 DAGScheduler: INFO: Submitting 2 missing tasks from ResultStage 9 (MapPartitionsRDD[49] at mapPartitions at ContextRDD.scala:217) (first 15 tasks are for partitions Vector(0, 1)) 2020-01-07 21:47:48 TaskSchedulerImpl: INFO: Adding task set 9.0 with 2 tasks 2020-01-07 21:47:48 TaskSetManager: INFO: Starting task 0.0 in stage 9.0 (TID 44, localhost, executor driver, partition 0, PROCESS_LOCAL, 8296 bytes) 2020-01-07 21:47:48 TaskSetManager: INFO: Starting task 1.0 in stage 9.0 (TID 45, localhost, executor driver, partition 1, PROCESS_LOCAL, 8296 bytes) 2020-01-07 21:47:48 Executor: INFO: Running task 0.0 in stage 9.0 (TID 44) 2020-01-07 21:47:48 Executor: INFO: Running task 1.0 in stage 9.0 (TID 45) 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:48 root: INFO: Index reader cache queries: 8 2020-01-07 21:47:48 root: INFO: Index reader cache hit rate: 0.625 2020-01-07 21:47:48 Executor: INFO: Finished task 1.0 in stage 9.0 (TID 45). 12135 bytes result sent to driver 2020-01-07 21:47:48 TaskSetManager: INFO: Finished task 1.0 in stage 9.0 (TID 45) in 213 ms on localhost (executor driver) (1/2) 2020-01-07 21:47:48 root: INFO: Index reader cache queries: 8 2020-01-07 21:47:48 root: INFO: Index reader cache hit rate: 0.625 2020-01-07 21:47:48 Executor: INFO: Finished task 0.0 in stage 9.0 (TID 44). 12135 bytes result sent to driver 2020-01-07 21:47:48 TaskSetManager: INFO: Finished task 0.0 in stage 9.0 (TID 44) in 225 ms on localhost (executor driver) (2/2) 2020-01-07 21:47:48 TaskSchedulerImpl: INFO: Removed TaskSet 9.0, whose tasks have all completed, from pool 2020-01-07 21:47:48 DAGScheduler: INFO: ResultStage 9 (runJob at RVD.scala:605) finished in 0.234 s 2020-01-07 21:47:48 DAGScheduler: INFO: Job 9 finished: runJob at RVD.scala:605, took 0.236894 s 2020-01-07 21:47:48 root: INFO: took 302.175ms 2020-01-07 21:47:48 root: INFO: interpreting non compilable node: Literal 2020-01-07 21:47:48 root: INFO: took 0.046ms 2020-01-07 21:47:48 root: INFO: interpreting non compilable node: TableCollect 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 root: INFO: encoder cache hit 2020-01-07 21:47:48 SparkContext: INFO: Starting job: collect at RVD.scala:676 2020-01-07 21:47:48 DAGScheduler: INFO: Got job 10 (collect at RVD.scala:676) with 1 output partitions 2020-01-07 21:47:48 DAGScheduler: INFO: Final stage: ResultStage 10 (collect at RVD.scala:676) 2020-01-07 21:47:48 DAGScheduler: INFO: Parents of final stage: List() 2020-01-07 21:47:48 DAGScheduler: INFO: Missing parents: List() 2020-01-07 21:47:48 DAGScheduler: INFO: Submitting ResultStage 10 (MapPartitionsRDD[55] at mapPartitions at ContextRDD.scala:217), which has no missing parents 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_21 stored as values in memory (estimated size 10.3 KB, free 365.4 MB) 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_21_piece0 stored as bytes in memory (estimated size 5.5 KB, free 365.4 MB) 2020-01-07 21:47:48 BlockManagerInfo: INFO: Added broadcast_21_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 5.5 KB, free: 366.2 MB) 2020-01-07 21:47:48 SparkContext: INFO: Created broadcast 21 from broadcast at DAGScheduler.scala:1161 2020-01-07 21:47:48 DAGScheduler: INFO: Submitting 1 missing tasks from ResultStage 10 (MapPartitionsRDD[55] at mapPartitions at ContextRDD.scala:217) (first 15 tasks are for partitions Vector(0)) 2020-01-07 21:47:48 TaskSchedulerImpl: INFO: Adding task set 10.0 with 1 tasks 2020-01-07 21:47:48 TaskSetManager: INFO: Starting task 0.0 in stage 10.0 (TID 46, localhost, executor driver, partition 0, PROCESS_LOCAL, 7653 bytes) 2020-01-07 21:47:48 Executor: INFO: Running task 0.0 in stage 10.0 (TID 46) 2020-01-07 21:47:48 Executor: INFO: Finished task 0.0 in stage 10.0 (TID 46). 3940 bytes result sent to driver 2020-01-07 21:47:48 TaskSetManager: INFO: Finished task 0.0 in stage 10.0 (TID 46) in 10 ms on localhost (executor driver) (1/1) 2020-01-07 21:47:48 TaskSchedulerImpl: INFO: Removed TaskSet 10.0, whose tasks have all completed, from pool 2020-01-07 21:47:48 DAGScheduler: INFO: ResultStage 10 (collect at RVD.scala:676) finished in 0.021 s 2020-01-07 21:47:48 DAGScheduler: INFO: Job 10 finished: collect at RVD.scala:676, took 0.024036 s 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 root: INFO: took 57.982ms 2020-01-07 21:47:48 root: INFO: interpreting non compilable node: TableAggregate 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 root: INFO: decoder cache miss (22 hits, 8 misses, 0.733 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeDecode_35. 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeDecode_35.apply 2020-01-07 21:47:48 root: INFO: instruction count: 27: is/hail/codegen/generated/C_etypeDecode_35.m0_DECODE_o_struct_of_o_array_of_o_struct_of_o_int32ANDo_array_of_r_int32ANDo_int32ANDo_int32ANDo_array_of_r_int32ENDEND_TO_o_tuple_of_o_array_of_o_tuple_of_o_callANDo_array_of_r_int32ENDEND 2020-01-07 21:47:48 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_35.m1_read_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 97: is/hail/codegen/generated/C_etypeDecode_35.m2_INPLACE_DECODE_o_array_of_o_struct_of_o_int32ANDo_array_of_r_int32ANDo_int32ANDo_int32ANDo_array_of_r_int32END_TO_o_array_of_o_tuple_of_o_int32ANDo_array_of_r_int32END 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_35.m3_INPLACE_DECODE_o_struct_of_o_int32ANDo_array_of_r_int32ANDo_int32ANDo_int32ANDo_array_of_r_int32END_TO_o_tuple_of_o_int32ANDo_array_of_r_int32END 2020-01-07 21:47:48 root: INFO: instruction count: 95: is/hail/codegen/generated/C_etypeDecode_35.m4_read_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_35.m5_INPLACE_DECODE_o_int32_TO_o_int32 2020-01-07 21:47:48 root: INFO: instruction count: 51: is/hail/codegen/generated/C_etypeDecode_35.m6_INPLACE_DECODE_o_array_of_r_int32_TO_o_array_of_r_int32 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_35.m7_INPLACE_DECODE_r_int32_TO_r_int32 2020-01-07 21:47:48 root: INFO: instruction count: 5: is/hail/codegen/generated/C_etypeDecode_35.m8_SKIP_o_int32 2020-01-07 21:47:48 root: INFO: instruction count: 25: is/hail/codegen/generated/C_etypeDecode_35.m9_SKIP_o_array_of_r_int32 2020-01-07 21:47:48 root: INFO: instruction count: 5: is/hail/codegen/generated/C_etypeDecode_35.m10_SKIP_r_int32 2020-01-07 21:47:48 root: INFO: optimize Compile: before: IR size 4: (InsertFields (Ref left) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref right)))) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.214ms, total 686.527ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.070ms, total 686.682ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.146ms, total 686.888ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.187ms, total 687.133ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.037ms, total 687.235ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.235ms, total 687.550ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 1.304ms, total 687.613ms, tagged coverage 68.2 2020-01-07 21:47:48 root: INFO: optimize Compile: after: IR size 4: (InsertFields (Ref left) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref right)))) 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C36. 2020-01-07 21:47:48 root: INFO: instruction count: 142: is/hail/codegen/generated/C36.apply 2020-01-07 21:47:48 root: INFO: instruction count: 22: is/hail/codegen/generated/C36.apply 2020-01-07 21:47:48 root: INFO: instruction count: 14: is/hail/codegen/generated/C36.setPartitionIndex 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C36.addPartitionRegion 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 root: INFO: optimize Compile: before: IR size 61: (Let __iruid_3120 (GetField rows (Literal Struct{rows:Array[Struct{}],global:Struct{}} )) (InsertFields (Ref global) ("__cols") (__cols (ArrayMap __iruid_3121 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_3120)) (I32 1)) (Let __iruid_3122 (GetField __iruid_3023 (ArrayRef (Literal Array[Struct{__iruid_3023:Struct{__iruid_3017:Int64,__iruid_3018:Int64,__iruid_3019:Int64,__iruid_3022:Struct{__iruid_3020:Int64,__iruid_3021:Float64}}}] ) (Ref __iruid_3121))) (Let __iruid_3123 (GetField __iruid_3022 (Ref __iruid_3122)) (InsertFields (ArrayRef (Ref __iruid_3120) (Ref __iruid_3121)) None (sample_qc (Let __iruid_3124 (MakeStruct (n_called (GetField __iruid_3017 (Ref __iruid_3122))) (n_not_called (GetField __iruid_3018 (Ref __iruid_3122))) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (GetField __iruid_3019 (Ref __iruid_3122))))) (Let __iruid_3125 (GetField n_called (Ref __iruid_3124)) (MakeStruct (dp_stats (Let __iruid_3126 (MakeStruct (n_def (GetField __iruid_3020 (Ref __iruid_3123))) (sum (GetField __iruid_3021 (Ref __iruid_3123)))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_3126)) (Cast Float64 (GetField n_def (Ref __iruid_3126)))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (Ref __iruid_3125)) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (Ref __iruid_3125) (GetField n_not_called (Ref __iruid_3124))) (GetField n_filtered (Ref __iruid_3124))))))))))))))))) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.383ms, total 698.523ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.176ms, total 698.761ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.416ms, total 699.227ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.953ms, total 700.248ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.153ms, total 700.465ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.036ms, total 701.544ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.261ms, total 701.864ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.090ms, total 702.022ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.419ms, total 702.512ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.185ms, total 703.766ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.143ms, total 703.969ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.904ms, total 704.918ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.203ms, total 705.206ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.073ms, total 705.342ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.272ms, total 705.654ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.633ms, total 706.336ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.131ms, total 706.520ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.887ms, total 707.447ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 9.365ms, total 707.502ms, tagged coverage 88.8 2020-01-07 21:47:48 root: INFO: optimize Compile: after: IR size 57: (InsertFields (Ref global) ("__cols") (__cols (ArrayMap __iruid_3172 (ArrayRange (I32 0) (I32 284) (I32 1)) (Let __iruid_3173 (GetField __iruid_3023 (ArrayRef (Literal Array[Struct{__iruid_3023:Struct{__iruid_3017:Int64,__iruid_3018:Int64,__iruid_3019:Int64,__iruid_3022:Struct{__iruid_3020:Int64,__iruid_3021:Float64}}}] ) (Ref __iruid_3172))) (Let __iruid_3174 (GetField __iruid_3022 (Ref __iruid_3173)) (InsertFields (ArrayRef (Literal Array[Struct{}] ) (Ref __iruid_3172)) None (sample_qc (Let __iruid_3175 (MakeStruct (n_called (GetField __iruid_3017 (Ref __iruid_3173))) (n_not_called (GetField __iruid_3018 (Ref __iruid_3173))) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (GetField __iruid_3019 (Ref __iruid_3173))))) (Let __iruid_3176 (GetField n_called (Ref __iruid_3175)) (MakeStruct (dp_stats (Let __iruid_3177 (MakeStruct (n_def (GetField __iruid_3020 (Ref __iruid_3174))) (sum (GetField __iruid_3021 (Ref __iruid_3174)))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_3177)) (Cast Float64 (GetField n_def (Ref __iruid_3177)))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (Ref __iruid_3176)) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (Ref __iruid_3176) (GetField n_not_called (Ref __iruid_3175))) (GetField n_filtered (Ref __iruid_3175)))))))))))))))) 2020-01-07 21:47:48 root: INFO: encoder cache miss (7 hits, 11 misses, 0.389) 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_38. 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_38.apply 2020-01-07 21:47:48 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_38.m0_ENCODE_o_tuple_of_o_array_of_o_tuple_of_ENDANDo_array_of_o_tuple_of_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDENDENDEND_TO_o_struct_of_o_array_of_o_struct_of_ENDANDo_array_of_o_struct_of_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDENDENDEND 2020-01-07 21:47:48 root: INFO: instruction count: 39: is/hail/codegen/generated/C_etypeEncode_38.m1_write_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 74: is/hail/codegen/generated/C_etypeEncode_38.m2_ENCODE_o_array_of_o_tuple_of_END_TO_o_array_of_o_struct_of_END 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_38.m3_ENCODE_o_tuple_of_END_TO_o_struct_of_END 2020-01-07 21:47:48 root: INFO: instruction count: 74: is/hail/codegen/generated/C_etypeEncode_38.m4_ENCODE_o_array_of_o_tuple_of_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDENDEND_TO_o_array_of_o_struct_of_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDENDEND 2020-01-07 21:47:48 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_38.m5_ENCODE_o_tuple_of_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDENDEND_TO_o_struct_of_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDENDEND 2020-01-07 21:47:48 root: INFO: instruction count: 19: is/hail/codegen/generated/C_etypeEncode_38.m6_write_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_38.m7_ENCODE_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDEND_TO_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDEND 2020-01-07 21:47:48 root: INFO: instruction count: 73: is/hail/codegen/generated/C_etypeEncode_38.m8_write_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_38.m9_ENCODE_o_int64_TO_o_int64 2020-01-07 21:47:48 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_38.m10_ENCODE_o_tuple_of_o_int64ANDo_float64END_TO_o_struct_of_o_int64ANDo_float64END 2020-01-07 21:47:48 root: INFO: instruction count: 39: is/hail/codegen/generated/C_etypeEncode_38.m11_write_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_38.m12_ENCODE_o_float64_TO_o_float64 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_22 stored as values in memory (estimated size 12.1 KB, free 365.4 MB) 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_22_piece0 stored as bytes in memory (estimated size 5.3 KB, free 365.4 MB) 2020-01-07 21:47:48 BlockManagerInfo: INFO: Added broadcast_22_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 5.3 KB, free: 366.1 MB) 2020-01-07 21:47:48 SparkContext: INFO: Created broadcast 22 from broadcast at SparkBackend.scala:21 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C37. 2020-01-07 21:47:48 root: INFO: instruction count: 44: is/hail/codegen/generated/C37.apply 2020-01-07 21:47:48 root: INFO: instruction count: 16: is/hail/codegen/generated/C37.apply 2020-01-07 21:47:48 root: INFO: instruction count: 611: is/hail/codegen/generated/C37.m1_method 2020-01-07 21:47:48 root: INFO: instruction count: 99: is/hail/codegen/generated/C37.m2_method 2020-01-07 21:47:48 root: INFO: instruction count: 43: is/hail/codegen/generated/C37.m3_method 2020-01-07 21:47:48 root: INFO: instruction count: 147: is/hail/codegen/generated/C37.m4_method 2020-01-07 21:47:48 root: INFO: instruction count: 148: is/hail/codegen/generated/C37.m5_method 2020-01-07 21:47:48 root: INFO: instruction count: 128: is/hail/codegen/generated/C37.m6_method 2020-01-07 21:47:48 root: INFO: instruction count: 77: is/hail/codegen/generated/C37.m7_method 2020-01-07 21:47:48 root: INFO: instruction count: 14: is/hail/codegen/generated/C37.setPartitionIndex 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C37.addPartitionRegion 2020-01-07 21:47:48 root: INFO: instruction count: 27: is/hail/codegen/generated/C37.m10_DECODE_o_struct_of_o_array_of_o_struct_of_ENDANDo_array_of_o_struct_of_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDENDENDEND_TO_o_tuple_of_o_array_of_o_tuple_of_ENDANDo_array_of_o_tuple_of_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDENDENDEND 2020-01-07 21:47:48 root: INFO: instruction count: 53: is/hail/codegen/generated/C37.m11_read_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 97: is/hail/codegen/generated/C37.m12_INPLACE_DECODE_o_array_of_o_struct_of_END_TO_o_array_of_o_tuple_of_END 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C37.m13_INPLACE_DECODE_o_struct_of_END_TO_o_tuple_of_END 2020-01-07 21:47:48 root: INFO: instruction count: 97: is/hail/codegen/generated/C37.m14_INPLACE_DECODE_o_array_of_o_struct_of_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDENDEND_TO_o_array_of_o_tuple_of_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDENDEND 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C37.m15_INPLACE_DECODE_o_struct_of_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDENDEND_TO_o_tuple_of_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDENDEND 2020-01-07 21:47:48 root: INFO: instruction count: 28: is/hail/codegen/generated/C37.m16_read_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C37.m17_INPLACE_DECODE_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDEND_TO_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDEND 2020-01-07 21:47:48 root: INFO: instruction count: 103: is/hail/codegen/generated/C37.m18_read_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C37.m19_INPLACE_DECODE_o_int64_TO_o_int64 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C37.m20_INPLACE_DECODE_o_struct_of_o_int64ANDo_float64END_TO_o_tuple_of_o_int64ANDo_float64END 2020-01-07 21:47:48 root: INFO: instruction count: 53: is/hail/codegen/generated/C37.m21_read_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C37.m22_INPLACE_DECODE_o_float64_TO_o_float64 2020-01-07 21:47:48 root: INFO: instruction count: 37: is/hail/codegen/generated/C37.addLiterals 2020-01-07 21:47:48 root: INFO: optimize Compile: before: IR size 4: (InsertFields (Literal Struct{} ) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)))) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.092ms, total 741.512ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.024ms, total 741.635ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.048ms, total 741.740ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.219ms, total 742.015ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.036ms, total 742.117ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.230ms, total 742.404ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 1.063ms, total 742.478ms, tagged coverage 61.0 2020-01-07 21:47:48 root: INFO: optimize Compile: after: IR size 4: (InsertFields (Literal Struct{} ) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)))) 2020-01-07 21:47:48 root: INFO: encoder cache miss (7 hits, 12 misses, 0.368) 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_40. 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_40.apply 2020-01-07 21:47:48 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_40.m0_ENCODE_o_tuple_of_o_tuple_of_ENDEND_TO_o_struct_of_o_struct_of_ENDEND 2020-01-07 21:47:48 root: INFO: instruction count: 19: is/hail/codegen/generated/C_etypeEncode_40.m1_write_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_40.m2_ENCODE_o_tuple_of_END_TO_o_struct_of_END 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_23 stored as values in memory (estimated size 56.0 B, free 365.4 MB) 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_23_piece0 stored as bytes in memory (estimated size 54.0 B, free 365.4 MB) 2020-01-07 21:47:48 BlockManagerInfo: INFO: Added broadcast_23_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 54.0 B, free: 366.1 MB) 2020-01-07 21:47:48 SparkContext: INFO: Created broadcast 23 from broadcast at SparkBackend.scala:21 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C39. 2020-01-07 21:47:48 root: INFO: instruction count: 80: is/hail/codegen/generated/C39.apply 2020-01-07 21:47:48 root: INFO: instruction count: 22: is/hail/codegen/generated/C39.apply 2020-01-07 21:47:48 root: INFO: instruction count: 14: is/hail/codegen/generated/C39.setPartitionIndex 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C39.addPartitionRegion 2020-01-07 21:47:48 root: INFO: instruction count: 27: is/hail/codegen/generated/C39.m3_DECODE_o_struct_of_o_struct_of_ENDEND_TO_o_tuple_of_o_tuple_of_ENDEND 2020-01-07 21:47:48 root: INFO: instruction count: 28: is/hail/codegen/generated/C39.m4_read_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C39.m5_INPLACE_DECODE_o_struct_of_END_TO_o_tuple_of_END 2020-01-07 21:47:48 root: INFO: instruction count: 28: is/hail/codegen/generated/C39.addLiterals 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 134 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 178 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 202 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 266 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 158 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 186 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 261 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 196 2020-01-07 21:47:48 root: INFO: Index reader cache queries: 8 2020-01-07 21:47:48 root: INFO: Index reader cache hit rate: 0.625 2020-01-07 21:47:48 root: INFO: Index reader cache queries: 8 2020-01-07 21:47:48 root: INFO: Index reader cache hit rate: 0.625 2020-01-07 21:47:48 root: INFO: Index reader cache queries: 8 2020-01-07 21:47:48 root: INFO: Index reader cache hit rate: 0.625 2020-01-07 21:47:48 BlockManagerInfo: INFO: Removed broadcast_21_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 5.5 KB, free: 366.2 MB) 2020-01-07 21:47:48 root: INFO: Index reader cache queries: 8 2020-01-07 21:47:48 root: INFO: Index reader cache hit rate: 0.625 2020-01-07 21:47:48 root: INFO: RegionPool: FREE: 1.6M allocated (1.6M blocks / 0 chunks), thread 99: Executor task launch worker for task 2 2020-01-07 21:47:48 root: INFO: optimize Compile: before: IR size 27: (InsertFields (Ref global) None (newColIdx (ArrayFilter __iruid_3127 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_3128 (GetField sample_qc (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_3127))) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (Ref __iruid_3128))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (Ref __iruid_3128)) (F64 0.97))) (False)))))) 2020-01-07 21:47:48 root: INFO: RegionPool: FREE: 136.5K allocated (136.5K blocks / 0 chunks), thread 193: Executor task launch worker for task 37 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.158ms, total 799.724ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.324ms, total 800.410ms 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 260 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 271 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 135 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 140 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 246 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 274 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 132 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 212 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 228 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.385ms, total 800.884ms 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 224 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 191 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 199 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 136 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 183 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 155 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 245 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 163 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 204 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 262 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 258 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 264 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 230 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 272 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 149 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 226 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 185 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 210 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 216 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 142 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 214 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 253 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 174 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 127 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 227 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 168 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 167 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 201 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 250 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.086ms, total 802.045ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.154ms, total 802.296ms 2020-01-07 21:47:48 BlockManagerInfo: INFO: Removed broadcast_20_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 33.8 KB, free: 366.2 MB) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.334ms, total 803.697ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.154ms, total 803.996ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.068ms, total 804.143ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.254ms, total 804.456ms 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 171 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 223 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 217 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 160 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 252 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 273 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 154 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 182 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 235 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 180 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 231 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 184 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 143 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 162 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 213 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 218 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 138 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 151 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 234 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 2.195ms, total 806.714ms 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 263 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 269 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 240 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.133ms, total 806.947ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.774ms, total 807.782ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.147ms, total 808.030ms 2020-01-07 21:47:48 BlockManagerInfo: INFO: Removed broadcast_16_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 5.5 KB, free: 366.2 MB) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.066ms, total 808.174ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.230ms, total 808.504ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.578ms, total 809.145ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.133ms, total 809.360ms 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 126 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 243 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 189 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 229 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 146 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 133 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 137 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 157 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 221 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 209 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 248 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 268 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 192 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 215 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 148 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 237 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 159 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 130 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 150 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 145 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 255 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 251 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 232 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 169 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 190 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 152 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 208 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 244 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 222 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 197 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 165 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 207 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 239 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.736ms, total 810.157ms 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 172 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 10.684ms, total 810.245ms, tagged coverage 83.4 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 233 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 200 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 256 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 275 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 144 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 257 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 188 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 156 2020-01-07 21:47:48 root: INFO: optimize Compile: after: IR size 27: (InsertFields (Ref global) None (newColIdx (ArrayFilter __iruid_3183 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_3184 (GetField sample_qc (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_3183))) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (Ref __iruid_3184))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (Ref __iruid_3184)) (F64 0.97))) (False)))))) 2020-01-07 21:47:48 BlockManagerInfo: INFO: Removed broadcast_12_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 5.5 KB, free: 366.2 MB) 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 187 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 193 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 131 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 195 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 219 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 267 2020-01-07 21:47:48 BlockManagerInfo: INFO: Removed broadcast_19_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 2.1 KB, free: 366.2 MB) 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 141 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 177 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 206 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 238 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 247 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 265 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 241 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 153 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 128 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 236 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 194 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 249 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 205 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 164 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 225 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 270 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 211 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C41. 2020-01-07 21:47:48 root: INFO: instruction count: 44: is/hail/codegen/generated/C41.apply 2020-01-07 21:47:48 root: INFO: instruction count: 16: is/hail/codegen/generated/C41.apply 2020-01-07 21:47:48 root: INFO: instruction count: 760: is/hail/codegen/generated/C41.m1_method 2020-01-07 21:47:48 root: INFO: instruction count: 177: is/hail/codegen/generated/C41.m2_method 2020-01-07 21:47:48 root: INFO: instruction count: 39: is/hail/codegen/generated/C41.m3_method 2020-01-07 21:47:48 root: INFO: instruction count: 14: is/hail/codegen/generated/C41.setPartitionIndex 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C41.addPartitionRegion 2020-01-07 21:47:48 BlockManagerInfo: INFO: Removed broadcast_15_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 33.8 KB, free: 366.2 MB) 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 220 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 175 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 254 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 179 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 203 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 259 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 173 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 176 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 242 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 129 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 147 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 166 2020-01-07 21:47:48 BlockManagerInfo: INFO: Removed broadcast_18_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 5.5 KB, free: 366.2 MB) 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 161 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 139 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 170 2020-01-07 21:47:48 root: INFO: optimize Compile: before: IR size 9: (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap __iruid_3129 (GetField newColIdx (Ref global)) (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_3129))))) 2020-01-07 21:47:48 BlockManagerInfo: INFO: Removed broadcast_14_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 2.1 KB, free: 366.2 MB) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.153ms, total 823.120ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.032ms, total 823.234ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.094ms, total 823.399ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.205ms, total 823.660ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.051ms, total 823.776ms 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 181 2020-01-07 21:47:48 ContextCleaner: INFO: Cleaned accumulator 198 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.335ms, total 824.162ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.101ms, total 824.345ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.028ms, total 824.432ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.078ms, total 824.555ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.165ms, total 824.767ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.046ms, total 824.880ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.279ms, total 825.211ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.088ms, total 825.384ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.028ms, total 825.469ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.077ms, total 825.592ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.170ms, total 825.814ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.046ms, total 825.922ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.268ms, total 826.241ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 3.351ms, total 826.313ms, tagged coverage 66.9 2020-01-07 21:47:48 root: INFO: optimize Compile: after: IR size 9: (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap __iruid_3188 (GetField newColIdx (Ref global)) (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_3188))))) 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C42. 2020-01-07 21:47:48 root: INFO: instruction count: 46: is/hail/codegen/generated/C42.apply 2020-01-07 21:47:48 root: INFO: instruction count: 22: is/hail/codegen/generated/C42.apply 2020-01-07 21:47:48 root: INFO: instruction count: 374: is/hail/codegen/generated/C42.m1_method 2020-01-07 21:47:48 root: INFO: instruction count: 14: is/hail/codegen/generated/C42.setPartitionIndex 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C42.addPartitionRegion 2020-01-07 21:47:48 root: INFO: encoder cache miss (7 hits, 13 misses, 0.350) 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_43. 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_43.apply 2020-01-07 21:47:48 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_43.m0_ENCODE_o_tuple_of_o_array_of_o_tuple_of_o_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDENDANDo_array_of_o_int32END_TO_o_struct_of_o_array_of_o_struct_of_o_struct_of_o_struct_of_o_float64ENDANDo_float64ENDENDANDo_array_of_o_int32END 2020-01-07 21:47:48 root: INFO: instruction count: 39: is/hail/codegen/generated/C_etypeEncode_43.m1_write_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 74: is/hail/codegen/generated/C_etypeEncode_43.m2_ENCODE_o_array_of_o_tuple_of_o_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDEND_TO_o_array_of_o_struct_of_o_struct_of_o_struct_of_o_float64ENDANDo_float64ENDEND 2020-01-07 21:47:48 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_43.m3_ENCODE_o_tuple_of_o_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDEND_TO_o_struct_of_o_struct_of_o_struct_of_o_float64ENDANDo_float64ENDEND 2020-01-07 21:47:48 root: INFO: instruction count: 19: is/hail/codegen/generated/C_etypeEncode_43.m4_write_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_43.m5_ENCODE_o_tuple_of_o_tuple_of_o_float64ENDANDo_float64END_TO_o_struct_of_o_struct_of_o_float64ENDANDo_float64END 2020-01-07 21:47:48 root: INFO: instruction count: 37: is/hail/codegen/generated/C_etypeEncode_43.m6_write_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_43.m7_ENCODE_o_tuple_of_o_float64END_TO_o_struct_of_o_float64END 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_43.m8_write_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_43.m9_ENCODE_o_float64_TO_o_float64 2020-01-07 21:47:48 root: INFO: instruction count: 76: is/hail/codegen/generated/C_etypeEncode_43.m10_ENCODE_o_array_of_o_int32_TO_o_array_of_o_int32 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_43.m11_ENCODE_o_int32_TO_o_int32 2020-01-07 21:47:48 root: INFO: decoder cache miss (24 hits, 9 misses, 0.727 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeDecode_44. 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeDecode_44.apply 2020-01-07 21:47:48 root: INFO: instruction count: 27: is/hail/codegen/generated/C_etypeDecode_44.m0_DECODE_o_struct_of_o_array_of_o_struct_of_o_struct_of_o_struct_of_o_float64ENDANDo_float64ENDENDANDo_array_of_o_int32END_TO_o_tuple_of_o_array_of_o_tuple_of_o_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDENDANDo_array_of_o_int32END 2020-01-07 21:47:48 root: INFO: instruction count: 53: is/hail/codegen/generated/C_etypeDecode_44.m1_read_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 97: is/hail/codegen/generated/C_etypeDecode_44.m2_INPLACE_DECODE_o_array_of_o_struct_of_o_struct_of_o_struct_of_o_float64ENDANDo_float64ENDEND_TO_o_array_of_o_tuple_of_o_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDEND 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_44.m3_INPLACE_DECODE_o_struct_of_o_struct_of_o_struct_of_o_float64ENDANDo_float64ENDEND_TO_o_tuple_of_o_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDEND 2020-01-07 21:47:48 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_44.m4_read_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_44.m5_INPLACE_DECODE_o_struct_of_o_struct_of_o_float64ENDANDo_float64END_TO_o_tuple_of_o_tuple_of_o_float64ENDANDo_float64END 2020-01-07 21:47:48 root: INFO: instruction count: 53: is/hail/codegen/generated/C_etypeDecode_44.m6_read_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_44.m7_INPLACE_DECODE_o_struct_of_o_float64END_TO_o_tuple_of_o_float64END 2020-01-07 21:47:48 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_44.m8_read_fields_group_0 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_44.m9_INPLACE_DECODE_o_float64_TO_o_float64 2020-01-07 21:47:48 root: INFO: instruction count: 97: is/hail/codegen/generated/C_etypeDecode_44.m10_INPLACE_DECODE_o_array_of_o_int32_TO_o_array_of_o_int32 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_44.m11_INPLACE_DECODE_o_int32_TO_o_int32 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_24 stored as values in memory (estimated size 13.6 KB, free 365.7 MB) 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_24_piece0 stored as bytes in memory (estimated size 7.4 KB, free 365.7 MB) 2020-01-07 21:47:48 BlockManagerInfo: INFO: Added broadcast_24_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 7.4 KB, free: 366.2 MB) 2020-01-07 21:47:48 SparkContext: INFO: Created broadcast 24 from broadcast at SparkBackend.scala:21 2020-01-07 21:47:48 root: INFO: optimize Compile: before: IR size 2: (SelectFields () (Ref global)) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.051ms, total 851.828ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.014ms, total 851.916ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.062ms, total 852.022ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.148ms, total 852.212ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.025ms, total 852.301ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.168ms, total 852.509ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.810ms, total 852.582ms, tagged coverage 57.8 2020-01-07 21:47:48 root: INFO: optimize Compile: after: IR size 2: (SelectFields () (Ref global)) 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C45. 2020-01-07 21:47:48 root: INFO: instruction count: 26: is/hail/codegen/generated/C45.apply 2020-01-07 21:47:48 root: INFO: instruction count: 16: is/hail/codegen/generated/C45.apply 2020-01-07 21:47:48 root: INFO: instruction count: 14: is/hail/codegen/generated/C45.setPartitionIndex 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C45.addPartitionRegion 2020-01-07 21:47:48 root: INFO: encoder cache hit 2020-01-07 21:47:48 root: INFO: decoder cache hit 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_25 stored as values in memory (estimated size 2.0 KB, free 365.7 MB) 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_25_piece0 stored as bytes in memory (estimated size 946.0 B, free 365.7 MB) 2020-01-07 21:47:48 BlockManagerInfo: INFO: Added broadcast_25_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 946.0 B, free: 366.2 MB) 2020-01-07 21:47:48 SparkContext: INFO: Created broadcast 25 from broadcast at SparkBackend.scala:21 2020-01-07 21:47:48 root: INFO: optimize Compile: before: IR size 3: (Begin (InitOp2 0 (Count () () None) ()) (InitOp2 1 (Count () () None) ())) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.064ms, total 861.486ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.027ms, total 861.585ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.031ms, total 861.701ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.205ms, total 861.947ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.026ms, total 862.032ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.098ms, total 862.171ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.832ms, total 862.251ms, tagged coverage 54.2 2020-01-07 21:47:48 root: INFO: optimize Compile: after: IR size 3: (Begin (InitOp2 0 (Count () () None) ()) (InitOp2 1 (Count () () None) ())) 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C46. 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C46.apply 2020-01-07 21:47:48 root: INFO: instruction count: 14: is/hail/codegen/generated/C46.apply 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C46.newAggState 2020-01-07 21:47:48 root: INFO: instruction count: 17: is/hail/codegen/generated/C46.setAggState 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C46.getAggOffset 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C46.setNumSerialized 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C46.setSerializedAgg 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C46.getSerializedAgg 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C46.m7_create_states_group0 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C46.m8_new_state_group0 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C46.m9_create_states_group0 2020-01-07 21:47:48 root: INFO: instruction count: 23: is/hail/codegen/generated/C46.m10_load_group0 2020-01-07 21:47:48 root: INFO: instruction count: 23: is/hail/codegen/generated/C46.m11_store_group0 2020-01-07 21:47:48 root: INFO: instruction count: 14: is/hail/codegen/generated/C46.setPartitionIndex 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C46.addPartitionRegion 2020-01-07 21:47:48 root: INFO: optimize Compile: before: IR size 76: (Let __iruid_3130 (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Begin (ArrayFor __iruid_3131 (ArrayFilter __iruid_3132 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_3130)) (I32 1)) (ApplyUnaryPrimOp Bang (IsNA (ArrayRef (Ref __iruid_3130) (Ref __iruid_3132))))) (Let __iruid_3133 (ArrayRef (Ref __iruid_3130) (Ref __iruid_3131)) (Let __iruid_3135 (GetField GT (Ref __iruid_3133)) (Let __iruid_3136 (GetField AD (Ref __iruid_3133)) (Let __iruid_3137 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (Ref __iruid_3136) (I32 1)) (ArrayFold __iruid_3138 __iruid_3139 (ArrayFilter __iruid_3140 (Ref __iruid_3136) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_3140)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_3138) (Ref __iruid_3139)))) (Begin (SeqOp2 0 (Count () () None) ()) (If (ApplyUnaryPrimOp Bang (ApplySpecial `||` Boolean (ApplySpecial `||` Boolean (ApplySpecial `&&` Boolean (Apply isHomRef Boolean (Ref __iruid_3135)) (ApplyComparisonOp LTEQ (Cast Float64 (Ref __iruid_3137)) (F64 0.1))) (ApplySpecial `&&` Boolean (ApplySpecial `&&` Boolean (Apply isHet Boolean (Ref __iruid_3135)) (ApplyComparisonOp GTEQ (Cast Float64 (Ref __iruid_3137)) (F64 0.25))) (ApplyComparisonOp LTEQ (Cast Float64 (Ref __iruid_3137)) (F64 0.75)))) (ApplySpecial `&&` Boolean (Apply isHomVar Boolean (Ref __iruid_3135)) (ApplyComparisonOp GTEQ (Cast Float64 (Ref __iruid_3137)) (F64 0.9))))) (Begin (SeqOp2 1 (Count () () None) ())) (Begin)))))))))) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.441ms, total 865.997ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.391ms, total 866.457ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 1.031ms, total 867.544ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 2.127ms, total 869.732ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.261ms, total 870.081ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.681ms, total 871.811ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.370ms, total 872.249ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.306ms, total 872.621ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.488ms, total 873.157ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.621ms, total 874.826ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.256ms, total 875.145ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.348ms, total 876.541ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.358ms, total 876.978ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.256ms, total 877.289ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.493ms, total 877.834ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.676ms, total 879.554ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.243ms, total 879.862ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.247ms, total 881.155ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 15.662ms, total 881.215ms, tagged coverage 93.2 2020-01-07 21:47:48 root: INFO: optimize Compile: after: IR size 75: (Let __iruid_3210 (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Begin (ArrayFor __iruid_3211 (ArrayFilter __iruid_3212 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_3210)) (I32 1)) (ApplyUnaryPrimOp Bang (IsNA (ArrayRef (Ref __iruid_3210) (Ref __iruid_3212))))) (Let __iruid_3213 (ArrayRef (Ref __iruid_3210) (Ref __iruid_3211)) (Let __iruid_3214 (GetField GT (Ref __iruid_3213)) (Let __iruid_3215 (GetField AD (Ref __iruid_3213)) (Let __iruid_3216 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (Ref __iruid_3215) (I32 1)) (ArrayFold __iruid_3217 __iruid_3218 (ArrayFilter __iruid_3219 (Ref __iruid_3215) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_3219)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_3217) (Ref __iruid_3218)))) (Begin (SeqOp2 0 (Count () () None) ()) (If (ApplySpecial `||` Boolean (ApplySpecial `||` Boolean (ApplySpecial `&&` Boolean (Apply isHomRef Boolean (Ref __iruid_3214)) (ApplyComparisonOp LTEQ (Cast Float64 (Ref __iruid_3216)) (F64 0.1))) (ApplySpecial `&&` Boolean (ApplySpecial `&&` Boolean (Apply isHet Boolean (Ref __iruid_3214)) (ApplyComparisonOp GTEQ (Cast Float64 (Ref __iruid_3216)) (F64 0.25))) (ApplyComparisonOp LTEQ (Cast Float64 (Ref __iruid_3216)) (F64 0.75)))) (ApplySpecial `&&` Boolean (Apply isHomVar Boolean (Ref __iruid_3214)) (ApplyComparisonOp GTEQ (Cast Float64 (Ref __iruid_3216)) (F64 0.9)))) (Begin) (Begin (SeqOp2 1 (Count () () None) ()))))))))))) 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C47. 2020-01-07 21:47:48 root: INFO: instruction count: 68: is/hail/codegen/generated/C47.apply 2020-01-07 21:47:48 root: INFO: instruction count: 20: is/hail/codegen/generated/C47.apply 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C47.newAggState 2020-01-07 21:47:48 root: INFO: instruction count: 17: is/hail/codegen/generated/C47.setAggState 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C47.getAggOffset 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C47.setNumSerialized 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C47.setSerializedAgg 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C47.getSerializedAgg 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C47.m7_create_states_group0 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C47.m8_new_state_group0 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C47.m9_create_states_group0 2020-01-07 21:47:48 root: INFO: instruction count: 23: is/hail/codegen/generated/C47.m10_load_group0 2020-01-07 21:47:48 root: INFO: instruction count: 23: is/hail/codegen/generated/C47.m11_store_group0 2020-01-07 21:47:48 root: INFO: instruction count: 543: is/hail/codegen/generated/C47.m12_method 2020-01-07 21:47:48 root: INFO: instruction count: 314: is/hail/codegen/generated/C47.m13_method 2020-01-07 21:47:48 root: INFO: instruction count: 830: is/hail/codegen/generated/C47.m14_method 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C47.m15_method 2020-01-07 21:47:48 root: INFO: instruction count: 39: is/hail/codegen/generated/C47.m16_method 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C47.m17_method 2020-01-07 21:47:48 root: INFO: instruction count: 39: is/hail/codegen/generated/C47.m18_method 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C47.m19_method 2020-01-07 21:47:48 root: INFO: instruction count: 14: is/hail/codegen/generated/C47.setPartitionIndex 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C47.addPartitionRegion 2020-01-07 21:47:48 root: INFO: optimize Compile: before: IR size 1: (DeserializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Count () () None) (Count () () None))) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.040ms, total 896.238ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.010ms, total 896.334ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.018ms, total 896.387ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.138ms, total 896.557ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.015ms, total 896.613ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.033ms, total 896.676ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.516ms, total 896.710ms, tagged coverage 49.3 2020-01-07 21:47:48 root: INFO: optimize Compile: after: IR size 1: (DeserializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Count () () None) (Count () () None))) 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C48. 2020-01-07 21:47:48 root: INFO: instruction count: 33: is/hail/codegen/generated/C48.apply 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C48.apply 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C48.newAggState 2020-01-07 21:47:48 root: INFO: instruction count: 17: is/hail/codegen/generated/C48.setAggState 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C48.getAggOffset 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C48.setNumSerialized 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C48.setSerializedAgg 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C48.getSerializedAgg 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C48.m7_create_states_group0 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C48.m8_new_state_group0 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C48.m9_create_states_group0 2020-01-07 21:47:48 root: INFO: instruction count: 23: is/hail/codegen/generated/C48.m10_load_group0 2020-01-07 21:47:48 root: INFO: instruction count: 23: is/hail/codegen/generated/C48.m11_store_group0 2020-01-07 21:47:48 root: INFO: instruction count: 13: is/hail/codegen/generated/C48.m12_deserialize_aggs_group0 2020-01-07 21:47:48 root: INFO: instruction count: 14: is/hail/codegen/generated/C48.setPartitionIndex 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C48.addPartitionRegion 2020-01-07 21:47:48 root: INFO: optimize Compile: before: IR size 1: (SerializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Count () () None) (Count () () None))) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.030ms, total 898.735ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.008ms, total 898.793ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.009ms, total 898.834ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.115ms, total 898.978ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.012ms, total 899.031ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.027ms, total 899.089ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.420ms, total 899.123ms, tagged coverage 47.8 2020-01-07 21:47:48 root: INFO: optimize Compile: after: IR size 1: (SerializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Count () () None) (Count () () None))) 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C49. 2020-01-07 21:47:48 root: INFO: instruction count: 48: is/hail/codegen/generated/C49.apply 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C49.apply 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C49.newAggState 2020-01-07 21:47:48 root: INFO: instruction count: 17: is/hail/codegen/generated/C49.setAggState 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C49.getAggOffset 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C49.setNumSerialized 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C49.setSerializedAgg 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C49.getSerializedAgg 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C49.m7_create_states_group0 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C49.m8_new_state_group0 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C49.m9_create_states_group0 2020-01-07 21:47:48 root: INFO: instruction count: 23: is/hail/codegen/generated/C49.m10_load_group0 2020-01-07 21:47:48 root: INFO: instruction count: 23: is/hail/codegen/generated/C49.m11_store_group0 2020-01-07 21:47:48 root: INFO: instruction count: 13: is/hail/codegen/generated/C49.m12_serialize_aggs_group0 2020-01-07 21:47:48 root: INFO: instruction count: 23: is/hail/codegen/generated/C49.m13_store_group0 2020-01-07 21:47:48 root: INFO: instruction count: 14: is/hail/codegen/generated/C49.setPartitionIndex 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C49.addPartitionRegion 2020-01-07 21:47:48 root: INFO: optimize Compile: before: IR size 6: (Begin (DeserializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Count () () None) (Count () () None))) (DeserializeAggs 2 1 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Count () () None) (Count () () None))) (CombOp2 0 2 (Count () () None)) (CombOp2 1 3 (Count () () None)) (SerializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Count () () None) (Count () () None)))) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.045ms, total 901.363ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.025ms, total 901.437ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.033ms, total 901.502ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 7.961ms, total 909.498ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.025ms, total 909.570ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.098ms, total 909.701ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 8.427ms, total 909.743ms, tagged coverage 97.1 2020-01-07 21:47:48 root: INFO: optimize Compile: after: IR size 6: (Begin (DeserializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Count () () None) (Count () () None))) (DeserializeAggs 2 1 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Count () () None) (Count () () None))) (CombOp2 0 2 (Count () () None)) (CombOp2 1 3 (Count () () None)) (SerializeAggs 0 0 "{\"name\":\"BlockingBufferSpec\",\"blockSize\":32768,\"child\":{\"name\":\"StreamBlockBufferSpec\"}}" ( (Count () () None) (Count () () None)))) 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C50. 2020-01-07 21:47:48 root: INFO: instruction count: 18: is/hail/codegen/generated/C50.apply 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C50.apply 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C50.newAggState 2020-01-07 21:47:48 root: INFO: instruction count: 17: is/hail/codegen/generated/C50.setAggState 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C50.getAggOffset 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C50.setNumSerialized 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C50.setSerializedAgg 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C50.getSerializedAgg 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C50.m7_create_states_group0 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C50.m8_new_state_group0 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C50.m9_create_states_group0 2020-01-07 21:47:48 root: INFO: instruction count: 43: is/hail/codegen/generated/C50.m10_load_group0 2020-01-07 21:47:48 root: INFO: instruction count: 43: is/hail/codegen/generated/C50.m11_store_group0 2020-01-07 21:47:48 root: INFO: instruction count: 86: is/hail/codegen/generated/C50.m12_method 2020-01-07 21:47:48 root: INFO: instruction count: 13: is/hail/codegen/generated/C50.m13_deserialize_aggs_group0 2020-01-07 21:47:48 root: INFO: instruction count: 13: is/hail/codegen/generated/C50.m14_deserialize_aggs_group0 2020-01-07 21:47:48 root: INFO: instruction count: 13: is/hail/codegen/generated/C50.m15_serialize_aggs_group0 2020-01-07 21:47:48 root: INFO: instruction count: 43: is/hail/codegen/generated/C50.m16_store_group0 2020-01-07 21:47:48 root: INFO: instruction count: 14: is/hail/codegen/generated/C50.setPartitionIndex 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C50.addPartitionRegion 2020-01-07 21:47:48 root: INFO: optimize Compile: before: IR size 19: (Let __iruid_3189 (ResultOp2 0 ( (Count () () None) (Count () () None))) (MakeTuple (0) (Let __iruid_3134 (GetTupleElement 0 (Ref __iruid_3189)) (If (ApplyComparisonOp EQ (Ref __iruid_3134) (I64 0)) (NA Float64) (ApplyBinaryPrimOp FloatingPointDivide (Let __iruid_3141 (GetTupleElement 1 (Ref __iruid_3189)) (Cast Float64 (Ref __iruid_3141))) (Cast Float64 (Ref __iruid_3134))))))) 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.246ms, total 913.397ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.062ms, total 913.535ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.157ms, total 913.728ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.796ms, total 915.561ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.065ms, total 915.677ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.408ms, total 916.121ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.111ms, total 916.282ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.051ms, total 916.372ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.124ms, total 916.545ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.391ms, total 916.976ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.061ms, total 917.084ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.366ms, total 917.484ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.110ms, total 917.641ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.053ms, total 917.731ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.113ms, total 917.880ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.440ms, total 918.362ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.060ms, total 918.469ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.548ms, total 919.050ms 2020-01-07 21:47:48 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 5.949ms, total 919.098ms, tagged coverage 86.8 2020-01-07 21:47:48 root: INFO: optimize Compile: after: IR size 17: (Let __iruid_3225 (ResultOp2 0 ( (Count () () None) (Count () () None))) (MakeTuple (0) (Let __iruid_3226 (GetTupleElement 0 (Ref __iruid_3225)) (If (ApplyComparisonOp EQ (Ref __iruid_3226) (I64 0)) (NA Float64) (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (GetTupleElement 1 (Ref __iruid_3225))) (Cast Float64 (Ref __iruid_3226))))))) 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C51. 2020-01-07 21:47:48 root: INFO: instruction count: 84: is/hail/codegen/generated/C51.apply 2020-01-07 21:47:48 root: INFO: instruction count: 16: is/hail/codegen/generated/C51.apply 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C51.newAggState 2020-01-07 21:47:48 root: INFO: instruction count: 17: is/hail/codegen/generated/C51.setAggState 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C51.getAggOffset 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C51.setNumSerialized 2020-01-07 21:47:48 root: INFO: instruction count: 8: is/hail/codegen/generated/C51.setSerializedAgg 2020-01-07 21:47:48 root: INFO: instruction count: 7: is/hail/codegen/generated/C51.getSerializedAgg 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C51.m7_create_states_group0 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C51.m8_new_state_group0 2020-01-07 21:47:48 root: INFO: instruction count: 3: is/hail/codegen/generated/C51.m9_create_states_group0 2020-01-07 21:47:48 root: INFO: instruction count: 23: is/hail/codegen/generated/C51.m10_load_group0 2020-01-07 21:47:48 root: INFO: instruction count: 23: is/hail/codegen/generated/C51.m11_store_group0 2020-01-07 21:47:48 root: INFO: instruction count: 21: is/hail/codegen/generated/C51.m12_aggs_result_group0 2020-01-07 21:47:48 root: INFO: instruction count: 23: is/hail/codegen/generated/C51.m13_store_group0 2020-01-07 21:47:48 root: INFO: instruction count: 164: is/hail/codegen/generated/C51.m14_method 2020-01-07 21:47:48 root: INFO: instruction count: 39: is/hail/codegen/generated/C51.m15_method 2020-01-07 21:47:48 root: INFO: instruction count: 14: is/hail/codegen/generated/C51.setPartitionIndex 2020-01-07 21:47:48 root: INFO: instruction count: 6: is/hail/codegen/generated/C51.addPartitionRegion 2020-01-07 21:47:48 root: INFO: Aggregate: useTreeAggregate=false 2020-01-07 21:47:48 root: INFO: Aggregate: commutative=true 2020-01-07 21:47:48 root: INFO: RegionPool: REPORT_THRESHOLD: 281.3K allocated (281.3K blocks / 0 chunks), thread 28: Thread-5 2020-01-07 21:47:48 SparkContext: INFO: Starting job: runJob at RVD.scala:605 2020-01-07 21:47:48 DAGScheduler: INFO: Got job 11 (runJob at RVD.scala:605) with 2 output partitions 2020-01-07 21:47:48 DAGScheduler: INFO: Final stage: ResultStage 11 (runJob at RVD.scala:605) 2020-01-07 21:47:48 DAGScheduler: INFO: Parents of final stage: List() 2020-01-07 21:47:48 DAGScheduler: INFO: Missing parents: List() 2020-01-07 21:47:48 DAGScheduler: INFO: Submitting ResultStage 11 (MapPartitionsRDD[62] at mapPartitions at ContextRDD.scala:217), which has no missing parents 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_26 stored as values in memory (estimated size 77.5 KB, free 365.6 MB) 2020-01-07 21:47:48 MemoryStore: INFO: Block broadcast_26_piece0 stored as bytes in memory (estimated size 35.1 KB, free 365.6 MB) 2020-01-07 21:47:48 BlockManagerInfo: INFO: Added broadcast_26_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 35.1 KB, free: 366.2 MB) 2020-01-07 21:47:48 SparkContext: INFO: Created broadcast 26 from broadcast at DAGScheduler.scala:1161 2020-01-07 21:47:48 DAGScheduler: INFO: Submitting 2 missing tasks from ResultStage 11 (MapPartitionsRDD[62] at mapPartitions at ContextRDD.scala:217) (first 15 tasks are for partitions Vector(0, 1)) 2020-01-07 21:47:48 TaskSchedulerImpl: INFO: Adding task set 11.0 with 2 tasks 2020-01-07 21:47:48 TaskSetManager: INFO: Starting task 0.0 in stage 11.0 (TID 47, localhost, executor driver, partition 0, PROCESS_LOCAL, 8296 bytes) 2020-01-07 21:47:48 TaskSetManager: INFO: Starting task 1.0 in stage 11.0 (TID 48, localhost, executor driver, partition 1, PROCESS_LOCAL, 8296 bytes) 2020-01-07 21:47:48 Executor: INFO: Running task 1.0 in stage 11.0 (TID 48) 2020-01-07 21:47:48 Executor: INFO: Running task 0.0 in stage 11.0 (TID 47) 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:48 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:47:49 root: INFO: Index reader cache queries: 8 2020-01-07 21:47:49 root: INFO: Index reader cache hit rate: 0.625 2020-01-07 21:47:49 Executor: INFO: Finished task 0.0 in stage 11.0 (TID 47). 691 bytes result sent to driver 2020-01-07 21:47:49 TaskSetManager: INFO: Finished task 0.0 in stage 11.0 (TID 47) in 391 ms on localhost (executor driver) (1/2) 2020-01-07 21:47:49 root: INFO: Index reader cache queries: 8 2020-01-07 21:47:49 root: INFO: Index reader cache hit rate: 0.625 2020-01-07 21:47:49 Executor: INFO: Finished task 1.0 in stage 11.0 (TID 48). 691 bytes result sent to driver 2020-01-07 21:47:49 TaskSetManager: INFO: Finished task 1.0 in stage 11.0 (TID 48) in 443 ms on localhost (executor driver) (2/2) 2020-01-07 21:47:49 TaskSchedulerImpl: INFO: Removed TaskSet 11.0, whose tasks have all completed, from pool 2020-01-07 21:47:49 DAGScheduler: INFO: ResultStage 11 (runJob at RVD.scala:605) finished in 0.454 s 2020-01-07 21:47:49 DAGScheduler: INFO: Job 11 finished: runJob at RVD.scala:605, took 0.458584 s 2020-01-07 21:47:49 root: INFO: took 729.901ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation : 1.136s, total 1.392s, tagged coverage 5.0 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- Verify : 0.027ms, total 1.392s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable : 1.136s, total 1.392s, tagged coverage 5.0 2020-01-07 21:47:49 root: INFO: optimize InterpretNonCompilable, post Lowering: before: IR size 1: (F64 0.03638682601952377) 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.055ms, total 1.392s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.020ms, total 1.392s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.021ms, total 1.392s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 0.100ms, total 1.393s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.027ms, total 1.393s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 0.075ms, total 1.393s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 0.468ms, total 1.393s, tagged coverage 63.9 2020-01-07 21:47:49 root: INFO: optimize InterpretNonCompilable, post Lowering: after: IR size 1: (F64 0.03638682601952377) 2020-01-07 21:47:49 root: INFO: instruction count: 3: is/hail/codegen/generated/C52. 2020-01-07 21:47:49 root: INFO: instruction count: 53: is/hail/codegen/generated/C52.apply 2020-01-07 21:47:49 root: INFO: instruction count: 10: is/hail/codegen/generated/C52.apply 2020-01-07 21:47:49 root: INFO: instruction count: 14: is/hail/codegen/generated/C52.setPartitionIndex 2020-01-07 21:47:49 root: INFO: instruction count: 6: is/hail/codegen/generated/C52.addPartitionRegion 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Compile : 1.513ms, total 1.394s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InitializeCompiledFunction : 0.360ms, total 1.395s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- RunCompiledFunction : 0.020ms, total 1.395s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- ConvertToSafeValue : 0.077ms, total 1.395s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate : 1.395s, total 1.395s, tagged coverage 21.1 2020-01-07 21:47:49 root: INFO: Timer: all timings: 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 3.002ms, total 9.382ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 15.192ms, total 24.718ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 19.547ms, total 44.393ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 38.314ms, total 82.992ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 6.207ms, total 89.530ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 27.433ms, total 117.110ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 1.179ms, total 118.401ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 6.339ms, total 124.799ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 3.579ms, total 128.475ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 18.602ms, total 147.164ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.773ms, total 148.065ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 5.100ms, total 153.216ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.661ms, total 154.024ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.601ms, total 154.674ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 2.648ms, total 157.381ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 4.007ms, total 161.458ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.515ms, total 162.053ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 3.899ms, total 166.002ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 159.705ms, total 166.079ms, tagged coverage 98.7 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- Verify : 0.088ms, total 168.425ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- LoweringTransformation : 14.726ms, total 183.215ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable -- Verify : 0.497ms, total 183.814ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- LowerMatrixToTable : 15.526ms, total 183.860ms, tagged coverage 98.6 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 1.041ms, total 189.182ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 1.118ms, total 190.388ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 8.030ms, total 198.477ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 8.845ms, total 207.415ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.927ms, total 208.454ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 8.699ms, total 217.216ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 1.164ms, total 218.465ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 1.010ms, total 219.531ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 2.642ms, total 222.250ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 6.215ms, total 228.538ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.730ms, total 229.368ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 6.965ms, total 236.393ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.973ms, total 237.479ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.946ms, total 238.481ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 1.609ms, total 240.142ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 4.287ms, total 244.476ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.778ms, total 245.311ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 6.639ms, total 251.999ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 63.919ms, total 252.055ms, tagged coverage 98.0 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- Verify : 0.085ms, total 256.360ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.627ms, total 331.961ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.108ms, total 332.138ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.183ms, total 332.364ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.689ms, total 333.091ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.174ms, total 333.324ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.937ms, total 334.316ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.242ms, total 334.613ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.091ms, total 334.752ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.171ms, total 334.962ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.636ms, total 335.638ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.093ms, total 335.779ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.825ms, total 336.642ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.178ms, total 336.877ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.078ms, total 337.003ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.161ms, total 337.200ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.624ms, total 337.858ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.099ms, total 338.009ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.743ms, total 338.789ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 7.517ms, total 338.847ms, tagged coverage 88.6 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.214ms, total 686.527ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.070ms, total 686.682ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.146ms, total 686.888ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.187ms, total 687.133ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.037ms, total 687.235ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.235ms, total 687.550ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 1.304ms, total 687.613ms, tagged coverage 68.2 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.383ms, total 698.523ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.176ms, total 698.761ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.416ms, total 699.227ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.953ms, total 700.248ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.153ms, total 700.465ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.036ms, total 701.544ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.261ms, total 701.864ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.090ms, total 702.022ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.419ms, total 702.512ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.185ms, total 703.766ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.143ms, total 703.969ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.904ms, total 704.918ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.203ms, total 705.206ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.073ms, total 705.342ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.272ms, total 705.654ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.633ms, total 706.336ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.131ms, total 706.520ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.887ms, total 707.447ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 9.365ms, total 707.502ms, tagged coverage 88.8 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.092ms, total 741.512ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.024ms, total 741.635ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.048ms, total 741.740ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.219ms, total 742.015ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.036ms, total 742.117ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.230ms, total 742.404ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 1.063ms, total 742.478ms, tagged coverage 61.0 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.158ms, total 799.724ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.324ms, total 800.410ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.385ms, total 800.884ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.086ms, total 802.045ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.154ms, total 802.296ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.334ms, total 803.697ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.154ms, total 803.996ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.068ms, total 804.143ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.254ms, total 804.456ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 2.195ms, total 806.714ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.133ms, total 806.947ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.774ms, total 807.782ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.147ms, total 808.030ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.066ms, total 808.174ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.230ms, total 808.504ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.578ms, total 809.145ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.133ms, total 809.360ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.736ms, total 810.157ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 10.684ms, total 810.245ms, tagged coverage 83.4 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.153ms, total 823.120ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.032ms, total 823.234ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.094ms, total 823.399ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.205ms, total 823.660ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.051ms, total 823.776ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.335ms, total 824.162ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.101ms, total 824.345ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.028ms, total 824.432ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.078ms, total 824.555ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.165ms, total 824.767ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.046ms, total 824.880ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.279ms, total 825.211ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.088ms, total 825.384ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.028ms, total 825.469ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.077ms, total 825.592ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.170ms, total 825.814ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.046ms, total 825.922ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.268ms, total 826.241ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 3.351ms, total 826.313ms, tagged coverage 66.9 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.051ms, total 851.828ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.014ms, total 851.916ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.062ms, total 852.022ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.148ms, total 852.212ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.025ms, total 852.301ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.168ms, total 852.509ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.810ms, total 852.582ms, tagged coverage 57.8 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.064ms, total 861.486ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.027ms, total 861.585ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.031ms, total 861.701ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.205ms, total 861.947ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.026ms, total 862.032ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.098ms, total 862.171ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.832ms, total 862.251ms, tagged coverage 54.2 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.441ms, total 865.997ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.391ms, total 866.457ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 1.031ms, total 867.544ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 2.127ms, total 869.732ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.261ms, total 870.081ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.681ms, total 871.811ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.370ms, total 872.249ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.306ms, total 872.621ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.488ms, total 873.157ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.621ms, total 874.826ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.256ms, total 875.145ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.348ms, total 876.541ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.358ms, total 876.978ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.256ms, total 877.289ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.493ms, total 877.834ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.676ms, total 879.554ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.243ms, total 879.862ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.247ms, total 881.155ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 15.662ms, total 881.215ms, tagged coverage 93.2 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.040ms, total 896.238ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.010ms, total 896.334ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.018ms, total 896.387ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.138ms, total 896.557ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.015ms, total 896.613ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.033ms, total 896.676ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.516ms, total 896.710ms, tagged coverage 49.3 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.030ms, total 898.735ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.008ms, total 898.793ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.009ms, total 898.834ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.115ms, total 898.978ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.012ms, total 899.031ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.027ms, total 899.089ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 0.420ms, total 899.123ms, tagged coverage 47.8 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.045ms, total 901.363ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.025ms, total 901.437ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.033ms, total 901.502ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 7.961ms, total 909.498ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.025ms, total 909.570ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.098ms, total 909.701ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 8.427ms, total 909.743ms, tagged coverage 97.1 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.246ms, total 913.397ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.062ms, total 913.535ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.157ms, total 913.728ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.796ms, total 915.561ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.065ms, total 915.677ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.408ms, total 916.121ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.111ms, total 916.282ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.051ms, total 916.372ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.124ms, total 916.545ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.391ms, total 916.976ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.061ms, total 917.084ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.366ms, total 917.484ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.110ms, total 917.641ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.053ms, total 917.731ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.113ms, total 917.880ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.440ms, total 918.362ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.060ms, total 918.469ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.548ms, total 919.050ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation -- Optimize : 5.949ms, total 919.098ms, tagged coverage 86.8 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- LoweringTransformation : 1.136s, total 1.392s, tagged coverage 5.0 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable -- Verify : 0.027ms, total 1.392s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InterpretNonCompilable : 1.136s, total 1.392s, tagged coverage 5.0 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- FoldConstants : 0.055ms, total 1.392s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ExtractIntervalFilters : 0.020ms, total 1.392s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- Simplify : 0.021ms, total 1.392s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardLets : 0.100ms, total 1.393s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- ForwardRelationalLets : 0.027ms, total 1.393s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize -- PruneDeadFields : 0.075ms, total 1.393s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Optimize : 0.468ms, total 1.393s, tagged coverage 63.9 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- Compile : 1.513ms, total 1.394s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- InitializeCompiledFunction : 0.360ms, total 1.395s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- RunCompiledFunction : 0.020ms, total 1.395s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate -- ConvertToSafeValue : 0.077ms, total 1.395s 2020-01-07 21:47:49 root: INFO: Timer: Time taken for CompileAndEvaluate : 1.395s, total 1.395s, tagged coverage 21.1 2020-01-07 21:47:49 root: INFO: Timer: aggregate: 2020-01-07 21:47:49 root: INFO: Timer: Time taken for tag 'RunCompiledFunction' (1): 0.020ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for tag 'ConvertToSafeValue' (1): 0.077ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for tag 'InitializeCompiledFunction' (1): 0.360ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for tag 'Verify' (4): 0.697ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for tag 'Compile' (1): 1.513ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for tag 'ForwardRelationalLets' (32): 12.434ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for tag 'FoldConstants' (32): 12.943ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for tag 'LoweringTransformation' (1): 14.726ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for tag 'ExtractIntervalFilters' (32): 27.683ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for tag 'Simplify' (32): 43.571ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for tag 'PruneDeadFields' (32): 74.355ms 2020-01-07 21:47:49 root: INFO: Timer: Time taken for tag 'ForwardLets' (32): 106.512ms 2020-01-07 21:47:49 root: INFO: Timer: Fraction covered by a tagged leaf: 1.100s (21.1%) 2020-01-07 21:48:23 root: INFO: optimize Lowerer, initial IR: before: IR size 1418: (TableMapRows (MatrixToTableApply "{\"name\":\"LinearRegressionRowsSingle\",\"yFields\":[\"__y_0\"],\"xField\":\"__uid_40\",\"covFields\":[\"__cov0\"],\"rowBlockSize\":16,\"passThrough\":[]}" (MatrixRename () () ("__uid_41" "__uid_42") ("__y_0" "__cov0") () () ("__uid_43") ("__uid_40") (MatrixMapEntries (MatrixMapCols None (MatrixMapRows (MatrixMapCols None (MatrixMapCols () (MatrixMapEntries (MatrixMapCols None (MatrixFilterRows (MatrixFilterRows (MatrixMapRows (MatrixFilterEntries (MatrixFilterCols (MatrixMapRows (MatrixMapCols None (MatrixMapRows (MatrixMapCols None (MatrixAnnotateColsTable "__uid_3" (MatrixRead None False False "{\"name\":\"MatrixNativeReader\",\"path\":\"data/1kg.mt\",\"_spec\":null}") (TableKeyBy (Sample) False (TableRead None False "{\"name\":\"TextTableReader\",\"options\":{\"files\":[\"data/1kg_annotations.txt\"],\"typeMapStr\":{},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"impute\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"forceGZ\":false}}"))) (InsertFields (SelectFields (s) (Ref sa)) None (pheno (GetField __uid_3 (Ref sa))))) (InsertFields (SelectFields (locus alleles rsid qual filters info) (Ref va)) None (__uid_4 (GetField AC (ApplyAggOp CallStats ( (ArrayLen (GetField alleles (Ref va)))) ( (GetField GT (Ref g)))))) (__uid_5 (ArrayMap __uid_6 (ApplyIR `[*:]` Array[String] (GetField alleles (Ref va)) (I32 1)) (Let __uid_11 (ApplyIR __uid_1 Int32 (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (Ref __uid_6)) (If (ApplyComparisonOp EQ (Ref __uid_11) (I32 1)) (If (ApplySpecial `&&` Boolean (ApplyComparisonOp EQ (ApplyIR __uid_1 Int32 (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (Ref __uid_6)) (I32 1)) (ApplyIR __uid_2 Boolean (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (Ref __uid_6))) (I32 8) (I32 9)) (Ref __uid_11))))))) (InsertFields (SelectFields (s pheno) (Ref sa)) None (sample_qc (Let __uid_27 (MakeStruct (n_called (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (GetField GT (Ref g)))))))) (n_not_called (ApplyAggOp Sum () ( (Apply toInt64 Int64 (IsNA (GetField GT (Ref g))))))) (n_filtered (ApplyBinaryPrimOp Subtract (TableCount (MatrixRowsTable (MatrixMapRows (MatrixMapCols None (MatrixAnnotateColsTable "__uid_3" (MatrixRead None False False "{\"name\":\"MatrixNativeReader\",\"path\":\"data/1kg.mt\",\"_spec\":null}") (TableKeyBy (Sample) False (TableRead None False "{\"name\":\"TextTableReader\",\"options\":{\"files\":[\"data/1kg_annotations.txt\"],\"typeMapStr\":{},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"impute\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"forceGZ\":false}}"))) (InsertFields (SelectFields (s) (Ref sa)) None (pheno (GetField __uid_3 (Ref sa))))) (InsertFields (SelectFields (locus alleles rsid qual filters info) (Ref va)) None (__uid_4 (GetField AC (ApplyAggOp CallStats ( (ArrayLen (GetField alleles (Ref va)))) ( (GetField GT (Ref g)))))) (__uid_5 (ArrayMap __uid_6 (ApplyIR `[*:]` Array[String] (GetField alleles (Ref va)) (I32 1)) (Let __uid_11 (ApplyIR __uid_1 Int32 (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (Ref __uid_6)) (If (ApplyComparisonOp EQ (Ref __uid_11) (I32 1)) (If (ApplySpecial `&&` Boolean (ApplyComparisonOp EQ (ApplyIR __uid_1 Int32 (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (Ref __uid_6)) (I32 1)) (ApplyIR __uid_2 Boolean (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (Ref __uid_6))) (I32 8) (I32 9)) (Ref __uid_11))))))))) (ApplyAggOp Count () ()))) (n_hom_ref (ApplyAggOp Sum () ( (Apply toInt64 Int64 (Apply isHomRef Boolean (GetField GT (Ref g))))))) (n_het (ApplyAggOp Sum () ( (Apply toInt64 Int64 (Apply isHet Boolean (GetField GT (Ref g))))))) (n_singleton (ApplyAggOp Sum () ( (ApplyIR toInt64 Int64 (ApplyIR sum Int32 (ArrayFilter __uid_24 (ArrayMap __uid_23 (ArrayMap __uid_22 (ArrayRange (I32 0) (Apply ploidy Int32 (GetField GT (Ref g))) (I32 1)) (ApplyComparisonOp EQ (ApplyIR indexArray Int32 (GetField __uid_4 (Ref va)) (Apply `[]` Int32 (GetField GT (Ref g)) (Ref __uid_22))) (I32 1))) (Apply toInt32 Int32 (Ref __uid_23))) (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_24))))))))) (allele_type_counts (AggExplode __uid_26 False (ArrayMap __uid_25 (ArrayRange (I32 0) (Apply ploidy Int32 (GetField GT (Ref g))) (I32 1)) (If (ApplyComparisonOp GT (Apply `[]` Int32 (GetField GT (Ref g)) (Ref __uid_25)) (I32 0)) (ApplyIR indexArray Int32 (GetField __uid_5 (Ref va)) (ApplyBinaryPrimOp Subtract (Apply `[]` Int32 (GetField GT (Ref g)) (Ref __uid_25)) (I32 1))) (NA Int32))) (AggGroupBy False (Ref __uid_26) (ApplyAggOp Count () ()))))) (Let __uid_28 (MakeStruct (dp_stats (SelectFields (mean stdev min max) (AggLet __uid_agg16 False (ApplyIR toFloat64 Float64 (GetField DP (Ref g))) (Let __uid_17 (MakeStruct (n_def (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_agg16))))))) (sum (ApplyAggOp Sum () ( (Ref __uid_agg16)))) (sumsq (ApplyAggOp Sum () ( (Apply `**` Float64 (Ref __uid_agg16) (ApplyIR toFloat64 Float64 (I32 2)))))) (min (ApplyAggOp Min () ( (Ref __uid_agg16)))) (max (ApplyAggOp Max () ( (Ref __uid_agg16))))) (Let __uid_18 (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __uid_17)) (ApplyIR toFloat64 Float64 (GetField n_def (Ref __uid_17)))) (MakeStruct (mean (Ref __uid_18)) (stdev (Apply sqrt Float64 (ApplyBinaryPrimOp FloatingPointDivide (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Subtract (GetField sumsq (Ref __uid_17)) (ApplyBinaryPrimOp Multiply (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (I32 2)) (Ref __uid_18)) (GetField sum (Ref __uid_17)))) (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (GetField n_def (Ref __uid_17))) (Apply `**` Float64 (Ref __uid_18) (ApplyIR toFloat64 Float64 (I32 2))))) (ApplyIR toFloat64 Float64 (GetField n_def (Ref __uid_17)))))) (min (GetField min (Ref __uid_17))) (max (GetField max (Ref __uid_17))) (n (GetField n_def (Ref __uid_17))) (sum (GetField sum (Ref __uid_17))))))))) (gq_stats (SelectFields (mean stdev min max) (AggLet __uid_agg19 False (ApplyIR toFloat64 Float64 (GetField GQ (Ref g))) (Let __uid_20 (MakeStruct (n_def (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_agg19))))))) (sum (ApplyAggOp Sum () ( (Ref __uid_agg19)))) (sumsq (ApplyAggOp Sum () ( (Apply `**` Float64 (Ref __uid_agg19) (ApplyIR toFloat64 Float64 (I32 2)))))) (min (ApplyAggOp Min () ( (Ref __uid_agg19)))) (max (ApplyAggOp Max () ( (Ref __uid_agg19))))) (Let __uid_21 (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __uid_20)) (ApplyIR toFloat64 Float64 (GetField n_def (Ref __uid_20)))) (MakeStruct (mean (Ref __uid_21)) (stdev (Apply sqrt Float64 (ApplyBinaryPrimOp FloatingPointDivide (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Subtract (GetField sumsq (Ref __uid_20)) (ApplyBinaryPrimOp Multiply (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (I32 2)) (Ref __uid_21)) (GetField sum (Ref __uid_20)))) (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (GetField n_def (Ref __uid_20))) (Apply `**` Float64 (Ref __uid_21) (ApplyIR toFloat64 Float64 (I32 2))))) (ApplyIR toFloat64 Float64 (GetField n_def (Ref __uid_20)))))) (min (GetField min (Ref __uid_20))) (max (GetField max (Ref __uid_20))) (n (GetField n_def (Ref __uid_20))) (sum (GetField sum (Ref __uid_20))))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (GetField n_called (Ref __uid_27))) (ApplyIR toFloat64 Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (GetField n_called (Ref __uid_27)) (GetField n_not_called (Ref __uid_27))) (GetField n_filtered (Ref __uid_27)))))) (n_called (GetField n_called (Ref __uid_27))) (n_not_called (GetField n_not_called (Ref __uid_27))) (n_filtered (GetField n_filtered (Ref __uid_27))) (n_hom_ref (GetField n_hom_ref (Ref __uid_27))) (n_het (GetField n_het (Ref __uid_27))) (n_hom_var (ApplyBinaryPrimOp Subtract (ApplyBinaryPrimOp Subtract (GetField n_called (Ref __uid_27)) (GetField n_hom_ref (Ref __uid_27))) (GetField n_het (Ref __uid_27)))) (n_non_ref (ApplyBinaryPrimOp Subtract (GetField n_called (Ref __uid_27)) (GetField n_hom_ref (Ref __uid_27)))) (n_singleton (GetField n_singleton (Ref __uid_27))) (n_snp (ApplyBinaryPrimOp Add (ApplyIR get Int64 (GetField allele_type_counts (Ref __uid_27)) (I32 8) (ApplyIR toInt64 Int64 (I32 0))) (ApplyIR get Int64 (GetField allele_type_counts (Ref __uid_27)) (I32 9) (ApplyIR toInt64 Int64 (I32 0))))) (n_insertion (ApplyIR get Int64 (GetField allele_type_counts (Ref __uid_27)) (I32 3) (ApplyIR toInt64 Int64 (I32 0)))) (n_deletion (ApplyIR get Int64 (GetField allele_type_counts (Ref __uid_27)) (I32 4) (ApplyIR toInt64 Int64 (I32 0)))) (n_transition (ApplyIR get Int64 (GetField allele_type_counts (Ref __uid_27)) (I32 8) (ApplyIR toInt64 Int64 (I32 0)))) (n_transversion (ApplyIR get Int64 (GetField allele_type_counts (Ref __uid_27)) (I32 9) (ApplyIR toInt64 Int64 (I32 0)))) (n_star (ApplyIR get Int64 (GetField allele_type_counts (Ref __uid_27)) (I32 6) (ApplyIR toInt64 Int64 (I32 0))))) (InsertFields (Ref __uid_28) None (r_ti_tv (If (ApplyComparisonOp NEQ (GetField n_transversion (Ref __uid_28)) (ApplyIR toInt64 Int64 (I32 0))) (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (GetField n_transition (Ref __uid_28))) (ApplyIR toFloat64 Float64 (GetField n_transversion (Ref __uid_28)))) (NA Float64))) (r_het_hom_var (If (ApplyComparisonOp NEQ (GetField n_hom_var (Ref __uid_28)) (ApplyIR toInt64 Int64 (I32 0))) (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (GetField n_het (Ref __uid_28))) (ApplyIR toFloat64 Float64 (GetField n_hom_var (Ref __uid_28)))) (NA Float64))) (r_insertion_deletion (If (ApplyComparisonOp NEQ (GetField n_deletion (Ref __uid_28)) (ApplyIR toInt64 Int64 (I32 0))) (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (GetField n_insertion (Ref __uid_28))) (ApplyIR toFloat64 Float64 (GetField n_deletion (Ref __uid_28)))) (NA Float64))))))))) (SelectFields (locus alleles rsid qual filters info) (MakeStruct (locus (GetField locus (Ref va))) (alleles (GetField alleles (Ref va))) (rsid (GetField rsid (Ref va))) (qual (GetField qual (Ref va))) (filters (GetField filters (Ref va))) (info (GetField info (Ref va))) (__uid_4 (GetField __uid_4 (Ref va))) (__uid_5 (GetField __uid_5 (Ref va)))))) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (GetField sample_qc (Ref sa)))) (ApplyIR toFloat64 Float64 (I32 4))) (ApplyComparisonOp GTEQ (GetField call_rate (GetField sample_qc (Ref sa))) (F64 0.97))) (False))) (Coalesce (ApplySpecial `||` Boolean (ApplySpecial `||` Boolean (ApplySpecial `&&` Boolean (Apply isHomRef Boolean (GetField GT (Ref g))) (ApplyComparisonOp LTEQ (ApplyIR toFloat64 Float64 (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR indexArray Int32 (GetField AD (Ref g)) (I32 1)) (ApplyIR sum Int32 (ArrayFilter __uid_29 (GetField AD (Ref g)) (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_29))))))) (F64 0.1))) (ApplySpecial `&&` Boolean (ApplySpecial `&&` Boolean (Apply isHet Boolean (GetField GT (Ref g))) (ApplyComparisonOp GTEQ (ApplyIR toFloat64 Float64 (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR indexArray Int32 (GetField AD (Ref g)) (I32 1)) (ApplyIR sum Int32 (ArrayFilter __uid_29 (GetField AD (Ref g)) (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_29))))))) (F64 0.25))) (ApplyComparisonOp LTEQ (ApplyIR toFloat64 Float64 (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR indexArray Int32 (GetField AD (Ref g)) (I32 1)) (ApplyIR sum Int32 (ArrayFilter __uid_29 (GetField AD (Ref g)) (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_29))))))) (F64 0.75)))) (ApplySpecial `&&` Boolean (Apply isHomVar Boolean (GetField GT (Ref g))) (ApplyComparisonOp GTEQ (ApplyIR toFloat64 Float64 (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR indexArray Int32 (GetField AD (Ref g)) (I32 1)) (ApplyIR sum Int32 (ArrayFilter __uid_29 (GetField AD (Ref g)) (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_29))))))) (F64 0.9)))) (False))) (AggLet __cse_21 False (GetField GT (Ref g)) (InsertFields (SelectFields (locus alleles rsid qual filters info) (Ref va)) None (variant_qc (Let __uid_37 (MakeStruct (n_called (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __cse_21))))))) (n_not_called (ApplyAggOp Sum () ( (Apply toInt64 Int64 (IsNA (Ref __cse_21)))))) (n_filtered (ApplyBinaryPrimOp Subtract (TableCount (MatrixColsTable (MatrixFilterEntries (MatrixFilterCols (MatrixMapRows (MatrixMapCols None (MatrixMapRows (MatrixMapCols None (MatrixAnnotateColsTable "__uid_3" (MatrixRead None False False "{\"name\":\"MatrixNativeReader\",\"path\":\"data/1kg.mt\",\"_spec\":null}") (TableKeyBy (Sample) False (TableRead None False "{\"name\":\"TextTableReader\",\"options\":{\"files\":[\"data/1kg_annotations.txt\"],\"typeMapStr\":{},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"impute\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"forceGZ\":false}}"))) (InsertFields (SelectFields (s) (Ref sa)) None (pheno (GetField __uid_3 (Ref sa))))) (InsertFields (SelectFields (locus alleles rsid qual filters info) (Ref va)) None (__uid_4 (GetField AC (ApplyAggOp CallStats ( (ArrayLen (GetField alleles (Ref va)))) ( (GetField GT (Ref g)))))) (__uid_5 (ArrayMap __uid_6 (ApplyIR `[*:]` Array[String] (GetField alleles (Ref va)) (I32 1)) (Let __uid_11 (ApplyIR __uid_1 Int32 (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (Ref __uid_6)) (If (ApplyComparisonOp EQ (Ref __uid_11) (I32 1)) (If (ApplySpecial `&&` Boolean (ApplyComparisonOp EQ (ApplyIR __uid_1 Int32 (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (Ref __uid_6)) (I32 1)) (ApplyIR __uid_2 Boolean (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (Ref __uid_6))) (I32 8) (I32 9)) (Ref __uid_11))))))) (AggLet __cse_24 False (GetField GT (Ref g)) (Let __cse_37 (ApplyIR toInt64 Int64 (I32 0)) (InsertFields (SelectFields (s pheno) (Ref sa)) None (sample_qc (Let __uid_27 (MakeStruct (n_called (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __cse_24))))))) (n_not_called (ApplyAggOp Sum () ( (Apply toInt64 Int64 (IsNA (Ref __cse_24)))))) (n_filtered (ApplyBinaryPrimOp Subtract (TableCount (MatrixRowsTable (MatrixMapRows (MatrixMapCols None (MatrixAnnotateColsTable "__uid_3" (MatrixRead None False False "{\"name\":\"MatrixNativeReader\",\"path\":\"data/1kg.mt\",\"_spec\":null}") (TableKeyBy (Sample) False (TableRead None False "{\"name\":\"TextTableReader\",\"options\":{\"files\":[\"data/1kg_annotations.txt\"],\"typeMapStr\":{},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"impute\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"forceGZ\":false}}"))) (InsertFields (SelectFields (s) (Ref sa)) None (pheno (GetField __uid_3 (Ref sa))))) (Let __cse_25 (GetField alleles (Ref va)) (InsertFields (SelectFields (locus alleles rsid qual filters info) (Ref va)) None (__uid_4 (GetField AC (ApplyAggOp CallStats ( (ArrayLen (GetField alleles (Ref va)))) ( (GetField GT (Ref g)))))) (__uid_5 (ArrayMap __uid_6 (ApplyIR `[*:]` Array[String] (Ref __cse_25) (I32 1)) (Let __uid_11 (ApplyIR __uid_1 Int32 (ApplyIR indexArray String (Ref __cse_25) (I32 0)) (Ref __uid_6)) (If (ApplyComparisonOp EQ (Ref __uid_11) (I32 1)) (Let __cse_26 (ApplyIR indexArray String (GetField alleles (Ref va)) (I32 0)) (If (ApplySpecial `&&` Boolean (ApplyComparisonOp EQ (ApplyIR __uid_1 Int32 (Ref __cse_26) (Ref __uid_6)) (I32 1)) (ApplyIR __uid_2 Boolean (Ref __cse_26) (Ref __uid_6))) (I32 8) (I32 9))) (Ref __uid_11)))))))))) (ApplyAggOp Count () ()))) (n_hom_ref (ApplyAggOp Sum () ( (Apply toInt64 Int64 (Apply isHomRef Boolean (Ref __cse_24)))))) (n_het (ApplyAggOp Sum () ( (Apply toInt64 Int64 (Apply isHet Boolean (Ref __cse_24)))))) (n_singleton (ApplyAggOp Sum () ( (ApplyIR toInt64 Int64 (ApplyIR sum Int32 (ArrayFilter __uid_24 (ArrayMap __uid_23 (ArrayMap __uid_22 (ArrayRange (I32 0) (Apply ploidy Int32 (Ref __cse_24)) (I32 1)) (ApplyComparisonOp EQ (ApplyIR indexArray Int32 (GetField __uid_4 (Ref va)) (Apply `[]` Int32 (Ref __cse_24) (Ref __uid_22))) (I32 1))) (Apply toInt32 Int32 (Ref __uid_23))) (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_24))))))))) (allele_type_counts (AggExplode __uid_26 False (ArrayMap __uid_25 (ArrayRange (I32 0) (Apply ploidy Int32 (Ref __cse_24)) (I32 1)) (If (ApplyComparisonOp GT (Apply `[]` Int32 (Ref __cse_24) (Ref __uid_25)) (I32 0)) (ApplyIR indexArray Int32 (GetField __uid_5 (Ref va)) (ApplyBinaryPrimOp Subtract (Apply `[]` Int32 (GetField GT (Ref g)) (Ref __uid_25)) (I32 1))) (NA Int32))) (AggGroupBy False (Ref __uid_26) (ApplyAggOp Count () ()))))) (Let __cse_31 (GetField n_called (Ref __uid_27)) (Let __cse_32 (GetField n_not_called (Ref __uid_27)) (Let __cse_33 (GetField n_filtered (Ref __uid_27)) (Let __cse_34 (GetField n_hom_ref (Ref __uid_27)) (Let __cse_35 (GetField n_het (Ref __uid_27)) (Let __cse_36 (GetField allele_type_counts (Ref __uid_27)) (Let __uid_28 (MakeStruct (dp_stats (SelectFields (mean stdev min max) (AggLet __uid_agg16 False (ApplyIR toFloat64 Float64 (GetField DP (Ref g))) (Let __uid_17 (MakeStruct (n_def (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_agg16))))))) (sum (ApplyAggOp Sum () ( (Ref __uid_agg16)))) (sumsq (ApplyAggOp Sum () ( (Apply `**` Float64 (Ref __uid_agg16) (ApplyIR toFloat64 Float64 (I32 2)))))) (min (ApplyAggOp Min () ( (Ref __uid_agg16)))) (max (ApplyAggOp Max () ( (Ref __uid_agg16))))) (Let __cse_27 (GetField sum (Ref __uid_17)) (Let __cse_28 (GetField n_def (Ref __uid_17)) (Let __uid_18 (ApplyBinaryPrimOp FloatingPointDivide (Ref __cse_27) (ApplyIR toFloat64 Float64 (Ref __cse_28))) (MakeStruct (mean (Ref __uid_18)) (stdev (Apply sqrt Float64 (ApplyBinaryPrimOp FloatingPointDivide (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Subtract (GetField sumsq (Ref __uid_17)) (ApplyBinaryPrimOp Multiply (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (I32 2)) (Ref __uid_18)) (Ref __cse_27))) (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (Ref __cse_28)) (Apply `**` Float64 (Ref __uid_18) (ApplyIR toFloat64 Float64 (I32 2))))) (ApplyIR toFloat64 Float64 (Ref __cse_28))))) (min (GetField min (Ref __uid_17))) (max (GetField max (Ref __uid_17))) (n (Ref __cse_28)) (sum (Ref __cse_27)))))))))) (gq_stats (SelectFields (mean stdev min max) (AggLet __uid_agg19 False (ApplyIR toFloat64 Float64 (GetField GQ (Ref g))) (Let __uid_20 (MakeStruct (n_def (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_agg19))))))) (sum (ApplyAggOp Sum () ( (Ref __uid_agg19)))) (sumsq (ApplyAggOp Sum () ( (Apply `**` Float64 (Ref __uid_agg19) (ApplyIR toFloat64 Float64 (I32 2)))))) (min (ApplyAggOp Min () ( (Ref __uid_agg19)))) (max (ApplyAggOp Max () ( (Ref __uid_agg19))))) (Let __cse_29 (GetField sum (Ref __uid_20)) (Let __cse_30 (GetField n_def (Ref __uid_20)) (Let __uid_21 (ApplyBinaryPrimOp FloatingPointDivide (Ref __cse_29) (ApplyIR toFloat64 Float64 (Ref __cse_30))) (MakeStruct (mean (Ref __uid_21)) (stdev (Apply sqrt Float64 (ApplyBinaryPrimOp FloatingPointDivide (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Subtract (GetField sumsq (Ref __uid_20)) (ApplyBinaryPrimOp Multiply (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (I32 2)) (Ref __uid_21)) (Ref __cse_29))) (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (Ref __cse_30)) (Apply `**` Float64 (Ref __uid_21) (ApplyIR toFloat64 Float64 (I32 2))))) (ApplyIR toFloat64 Float64 (Ref __cse_30))))) (min (GetField min (Ref __uid_20))) (max (GetField max (Ref __uid_20))) (n (Ref __cse_30)) (sum (Ref __cse_29)))))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (Ref __cse_31)) (ApplyIR toFloat64 Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (Ref __cse_31) (Ref __cse_32)) (Ref __cse_33))))) (n_called (Ref __cse_31)) (n_not_called (Ref __cse_32)) (n_filtered (Ref __cse_33)) (n_hom_ref (Ref __cse_34)) (n_het (Ref __cse_35)) (n_hom_var (ApplyBinaryPrimOp Subtract (ApplyBinaryPrimOp Subtract (Ref __cse_31) (Ref __cse_34)) (Ref __cse_35))) (n_non_ref (ApplyBinaryPrimOp Subtract (Ref __cse_31) (Ref __cse_34))) (n_singleton (GetField n_singleton (Ref __uid_27))) (n_snp (ApplyBinaryPrimOp Add (ApplyIR get Int64 (Ref __cse_36) (I32 8) (Ref __cse_37)) (ApplyIR get Int64 (Ref __cse_36) (I32 9) (Ref __cse_37)))) (n_insertion (ApplyIR get Int64 (Ref __cse_36) (I32 3) (Ref __cse_37))) (n_deletion (ApplyIR get Int64 (Ref __cse_36) (I32 4) (Ref __cse_37))) (n_transition (ApplyIR get Int64 (Ref __cse_36) (I32 8) (Ref __cse_37))) (n_transversion (ApplyIR get Int64 (Ref __cse_36) (I32 9) (Ref __cse_37))) (n_star (ApplyIR get Int64 (Ref __cse_36) (I32 6) (Ref __cse_37)))) (InsertFields (Ref __uid_28) None (r_ti_tv (If (ApplyComparisonOp NEQ (GetField n_transversion (Ref __uid_28)) (ApplyIR toInt64 Int64 (I32 0))) (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (GetField n_transition (Ref __uid_28))) (ApplyIR toFloat64 Float64 (GetField n_transversion (Ref __uid_28)))) (NA Float64))) (r_het_hom_var (If (ApplyComparisonOp NEQ (GetField n_hom_var (Ref __uid_28)) (ApplyIR toInt64 Int64 (I32 0))) (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (GetField n_het (Ref __uid_28))) (ApplyIR toFloat64 Float64 (GetField n_hom_var (Ref __uid_28)))) (NA Float64))) (r_insertion_deletion (If (ApplyComparisonOp NEQ (GetField n_deletion (Ref __uid_28)) (ApplyIR toInt64 Int64 (I32 0))) (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (GetField n_insertion (Ref __uid_28))) (ApplyIR toFloat64 Float64 (GetField n_deletion (Ref __uid_28)))) (NA Float64))))))))))))))))) (SelectFields (locus alleles rsid qual filters info) (MakeStruct (locus (GetField locus (Ref va))) (alleles (GetField alleles (Ref va))) (rsid (GetField rsid (Ref va))) (qual (GetField qual (Ref va))) (filters (GetField filters (Ref va))) (info (GetField info (Ref va))) (__uid_4 (GetField __uid_4 (Ref va))) (__uid_5 (GetField __uid_5 (Ref va)))))) (Let __cse_38 (GetField sample_qc (Ref sa)) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (Ref __cse_38))) (ApplyIR toFloat64 Float64 (I32 4))) (ApplyComparisonOp GTEQ (GetField call_rate (Ref __cse_38)) (F64 0.97))) (False)))) (Let __cse_40 (GetField GT (Ref g)) (Let __cse_39 (GetField AD (Ref g)) (Let __cse_41 (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR indexArray Int32 (Ref __cse_39) (I32 1)) (ApplyIR sum Int32 (ArrayFilter __uid_29 (Ref __cse_39) (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_29)))))) (Coalesce (ApplySpecial `||` Boolean (ApplySpecial `||` Boolean (ApplySpecial `&&` Boolean (Apply isHomRef Boolean (Ref __cse_40)) (ApplyComparisonOp LTEQ (ApplyIR toFloat64 Float64 (Ref __cse_41)) (F64 0.1))) (ApplySpecial `&&` Boolean (ApplySpecial `&&` Boolean (Apply isHet Boolean (Ref __cse_40)) (ApplyComparisonOp GTEQ (ApplyIR toFloat64 Float64 (Ref __cse_41)) (F64 0.25))) (ApplyComparisonOp LTEQ (ApplyIR toFloat64 Float64 (Ref __cse_41)) (F64 0.75)))) (ApplySpecial `&&` Boolean (Apply isHomVar Boolean (Ref __cse_40)) (ApplyComparisonOp GTEQ (ApplyIR toFloat64 Float64 (Ref __cse_41)) (F64 0.9)))) (False)))))))) (ApplyAggOp Count () ()))) (call_stats (ApplyAggOp CallStats ( (ArrayLen (GetField alleles (Ref va)))) ( (Ref __cse_21))))) (Let __cse_48 (GetField call_stats (Ref __uid_37)) (Let __cse_52 (GetField homozygote_count (Ref __cse_48)) (Let __cse_49 (GetField n_called (Ref __uid_37)) (Let __cse_50 (GetField n_not_called (Ref __uid_37)) (Let __cse_51 (GetField n_filtered (Ref __uid_37)) (Let __uid_38 (If (ApplyComparisonOp EQ (ArrayLen (GetField alleles (Ref va))) (I32 2)) (Let __cse_42 (GetField call_stats (Ref __uid_37)) (Let __cse_43 (GetField homozygote_count (Ref __cse_42)) (Apply hardy_weinberg_test Struct{het_freq_hwe:Float64,p_value:Float64} (ApplyIR indexArray Int32 (Ref __cse_43) (I32 0)) (ApplyBinaryPrimOp Subtract (ApplyIR indexArray Int32 (GetField AC (Ref __cse_42)) (I32 1)) (ApplyBinaryPrimOp Multiply (I32 2) (ApplyIR indexArray Int32 (Ref __cse_43) (I32 1)))) (ApplyIR indexArray Int32 (Ref __cse_43) (I32 1))))) (NA Struct{het_freq_hwe:Float64,p_value:Float64})) (MakeStruct (dp_stats (SelectFields (mean stdev min max) (AggLet __uid_agg31 False (ApplyIR toFloat64 Float64 (GetField DP (Ref g))) (Let __uid_32 (MakeStruct (n_def (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_agg31))))))) (sum (ApplyAggOp Sum () ( (Ref __uid_agg31)))) (sumsq (ApplyAggOp Sum () ( (Apply `**` Float64 (Ref __uid_agg31) (ApplyIR toFloat64 Float64 (I32 2)))))) (min (ApplyAggOp Min () ( (Ref __uid_agg31)))) (max (ApplyAggOp Max () ( (Ref __uid_agg31))))) (Let __cse_44 (GetField sum (Ref __uid_32)) (Let __cse_45 (GetField n_def (Ref __uid_32)) (Let __uid_33 (ApplyBinaryPrimOp FloatingPointDivide (Ref __cse_44) (ApplyIR toFloat64 Float64 (Ref __cse_45))) (MakeStruct (mean (Ref __uid_33)) (stdev (Apply sqrt Float64 (ApplyBinaryPrimOp FloatingPointDivide (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Subtract (GetField sumsq (Ref __uid_32)) (ApplyBinaryPrimOp Multiply (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (I32 2)) (Ref __uid_33)) (Ref __cse_44))) (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (Ref __cse_45)) (Apply `**` Float64 (Ref __uid_33) (ApplyIR toFloat64 Float64 (I32 2))))) (ApplyIR toFloat64 Float64 (Ref __cse_45))))) (min (GetField min (Ref __uid_32))) (max (GetField max (Ref __uid_32))) (n (Ref __cse_45)) (sum (Ref __cse_44)))))))))) (gq_stats (SelectFields (mean stdev min max) (AggLet __uid_agg34 False (ApplyIR toFloat64 Float64 (GetField GQ (Ref g))) (Let __uid_35 (MakeStruct (n_def (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_agg34))))))) (sum (ApplyAggOp Sum () ( (Ref __uid_agg34)))) (sumsq (ApplyAggOp Sum () ( (Apply `**` Float64 (Ref __uid_agg34) (ApplyIR toFloat64 Float64 (I32 2)))))) (min (ApplyAggOp Min () ( (Ref __uid_agg34)))) (max (ApplyAggOp Max () ( (Ref __uid_agg34))))) (Let __cse_46 (GetField sum (Ref __uid_35)) (Let __cse_47 (GetField n_def (Ref __uid_35)) (Let __uid_36 (ApplyBinaryPrimOp FloatingPointDivide (Ref __cse_46) (ApplyIR toFloat64 Float64 (Ref __cse_47))) (MakeStruct (mean (Ref __uid_36)) (stdev (Apply sqrt Float64 (ApplyBinaryPrimOp FloatingPointDivide (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Subtract (GetField sumsq (Ref __uid_35)) (ApplyBinaryPrimOp Multiply (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (I32 2)) (Ref __uid_36)) (Ref __cse_46))) (ApplyBinaryPrimOp Multiply (ApplyIR toFloat64 Float64 (Ref __cse_47)) (Apply `**` Float64 (Ref __uid_36) (ApplyIR toFloat64 Float64 (I32 2))))) (ApplyIR toFloat64 Float64 (Ref __cse_47))))) (min (GetField min (Ref __uid_35))) (max (GetField max (Ref __uid_35))) (n (Ref __cse_47)) (sum (Ref __cse_46)))))))))) (AC (GetField AC (Ref __cse_48))) (AF (GetField AF (Ref __cse_48))) (AN (GetField AN (Ref __cse_48))) (homozygote_count (Ref __cse_52)) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (ApplyIR toFloat64 Float64 (Ref __cse_49)) (ApplyIR toFloat64 Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (Ref __cse_49) (Ref __cse_50)) (Ref __cse_51))))) (n_called (Ref __cse_49)) (n_not_called (Ref __cse_50)) (n_filtered (Ref __cse_51)) (n_het (ApplyBinaryPrimOp Subtract (Ref __cse_49) (ApplyIR toInt64 Int64 (ApplyIR sum Int32 (ArrayFilter __uid_39 (Ref __cse_52) (ApplyUnaryPrimOp Bang (IsNA (Ref __uid_39)))))))) (n_non_ref (ApplyBinaryPrimOp Subtract (Ref __cse_49) (ApplyIR toInt64 Int64 (ApplyIR indexArray Int32 (Ref __cse_52) (I32 0))))) (het_freq_hwe (GetField het_freq_hwe (Ref __uid_38))) (p_value_hwe (GetField p_value (Ref __uid_38))))))))))))))) (Coalesce (ApplyComparisonOp GT (ApplyIR indexArray Float64 (GetField AF (GetField variant_qc (Ref va))) (I32 1)) (F64 0.01)) (False))) (Coalesce (ApplyComparisonOp GT (GetField p_value_hwe (GetField variant_qc (Ref va))) (F64 1.0E-6)) (False))) (InsertFields (SelectFields (s pheno sample_qc) (Ref sa)) None (__uid_41 (ApplyIR toFloat64 Float64 (GetField CaffeineConsumption (GetField pheno (Ref sa))))) (__uid_42 (F64 1.0)))) (InsertFields (SelectFields (GT AD DP GQ PL) (Ref g)) None (__uid_43 (ApplyIR toFloat64 Float64 (Apply nNonRefAlleles Int32 (GetField GT (Ref g))))))) (SelectFields (s pheno sample_qc __uid_41 __uid_42) (Ref sa))) (SelectFields (pheno sample_qc __uid_41 __uid_42) (SelectFields (s pheno sample_qc __uid_41 __uid_42) (Ref sa)))) (SelectFields (locus alleles) (MakeStruct (locus (GetField locus (Ref va))) (alleles (GetField alleles (Ref va))) (rsid (GetField rsid (Ref va))) (qual (GetField qual (Ref va))) (filters (GetField filters (Ref va))) (info (GetField info (Ref va))) (variant_qc (GetField variant_qc (Ref va)))))) (SelectFields (__uid_41 __uid_42) (SelectFields (pheno sample_qc __uid_41 __uid_42) (Ref sa)))) (SelectFields (__uid_43) (SelectFields (GT AD DP GQ PL __uid_43) (Ref g)))))) (InsertFields (SelectFields (locus alleles n sum_x y_transpose_x beta standard_error t_stat p_value) (Ref row)) None (y_transpose_x (ApplyIR indexArray Float64 (GetField y_transpose_x (Ref row)) (I32 0))) (beta (ApplyIR indexArray Float64 (GetField beta (Ref row)) (I32 0))) (standard_error (ApplyIR indexArray Float64 (GetField standard_error (Ref row)) (I32 0))) (t_stat (ApplyIR indexArray Float64 (GetField t_stat (Ref row)) (I32 0))) (p_value (ApplyIR indexArray Float64 (GetField p_value (Ref row)) (I32 0))))) 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 6.264ms, total 25.384ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 23.531ms, total 49.020ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- Simplify : 45.499ms, total 94.624ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 39.214ms, total 133.940ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 3.696ms, total 137.738ms 2020-01-07 21:48:23 root: INFO: Prune: InsertFields: eliminating field '__uid_4' 2020-01-07 21:48:23 root: INFO: Prune: InsertFields: eliminating field '__uid_5' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_hom_ref' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_het' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_singleton' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'allele_type_counts' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'sumsq' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'min' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'max' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'stdev' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'min' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'max' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'sum' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'gq_stats' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_called' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_not_called' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_filtered' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_hom_ref' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_het' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_hom_var' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_non_ref' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_singleton' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_snp' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_insertion' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_deletion' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_transition' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_transversion' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_star' 2020-01-07 21:48:23 root: INFO: Prune: InsertFields: eliminating field 'r_ti_tv' 2020-01-07 21:48:23 root: INFO: Prune: InsertFields: eliminating field 'r_het_hom_var' 2020-01-07 21:48:23 root: INFO: Prune: InsertFields: eliminating field 'r_insertion_deletion' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'rsid' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'qual' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'filters' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'info' 2020-01-07 21:48:23 root: INFO: Prune: InsertFields: eliminating field 'pheno' 2020-01-07 21:48:23 root: INFO: Prune: InsertFields: eliminating field '__uid_4' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_singleton' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'sumsq' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'min' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'max' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'stdev' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'min' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'max' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'sum' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'gq_stats' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_called' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_not_called' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_filtered' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_hom_ref' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_het' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_hom_var' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_non_ref' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_singleton' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_snp' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_insertion' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_deletion' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_transition' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_transversion' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_star' 2020-01-07 21:48:23 root: INFO: Prune: InsertFields: eliminating field 'r_ti_tv' 2020-01-07 21:48:23 root: INFO: Prune: InsertFields: eliminating field 'r_het_hom_var' 2020-01-07 21:48:23 root: INFO: Prune: InsertFields: eliminating field 'r_insertion_deletion' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'locus' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'alleles' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'rsid' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'qual' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'filters' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'info' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'dp_stats' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'gq_stats' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'AC' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'AN' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'homozygote_count' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'call_rate' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_called' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_not_called' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_filtered' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_het' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_non_ref' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'het_freq_hwe' 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 31.879ms, total 169.669ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 1.826ms, total 171.600ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 4.502ms, total 176.151ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- Simplify : 8.632ms, total 184.837ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 17.388ms, total 202.324ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.964ms, total 203.389ms 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_called' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_not_called' 2020-01-07 21:48:23 root: INFO: Prune: MakeStruct: eliminating field 'n_filtered' 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 10.012ms, total 213.460ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.583ms, total 214.118ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 1.368ms, total 215.531ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- Simplify : 1.628ms, total 217.222ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 6.688ms, total 223.956ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.545ms, total 224.566ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 8.556ms, total 233.173ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize : 214.147ms, total 233.261ms, tagged coverage 99.4 2020-01-07 21:48:23 root: INFO: optimize Lowerer, initial IR: after: IR size 293: (TableMapRows (MatrixToTableApply "{\"name\":\"LinearRegressionRowsSingle\",\"yFields\":[\"__y_0\"],\"xField\":\"__uid_40\",\"covFields\":[\"__cov0\"],\"rowBlockSize\":16,\"passThrough\":[]}" (MatrixRename () () ("__uid_41" "__uid_42") ("__y_0" "__cov0") () () ("__uid_43") ("__uid_40") (MatrixMapEntries (MatrixMapRows (MatrixMapCols () (MatrixMapEntries (MatrixMapCols None (MatrixFilterRows (MatrixMapRows (MatrixFilterEntries (MatrixFilterCols (MatrixMapRows (MatrixMapCols () (MatrixAnnotateColsTable "__uid_3" (MatrixRead Matrix{global:Struct{},col_key:[s],col:Struct{s:String},row_key:[[locus,alleles]],row:Struct{locus:Locus(GRCh37),alleles:Array[String]},entry:Struct{GT:Call,AD:Array[+Int32],DP:Int32}} False False "{\"name\":\"MatrixNativeReader\",\"path\":\"data/1kg.mt\",\"_spec\":null}") (TableKeyBy (Sample) False (TableRead Table{global:Struct{},key:[],row:Struct{Sample:String,CaffeineConsumption:Int32}} False "{\"name\":\"TextTableReader\",\"options\":{\"files\":[\"data/1kg_annotations.txt\"],\"typeMapStr\":{},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"impute\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"forceGZ\":false}}"))) (InsertFields (SelectFields () (Ref sa)) ("pheno" "sample_qc") (pheno (GetField __uid_3 (Ref sa))) (sample_qc (Let __iruid_5790 (MakeStruct (n_called (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (GetField GT (Ref g)))))))) (n_not_called (ApplyAggOp Sum () ( (Apply toInt64 Int64 (IsNA (GetField GT (Ref g))))))) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (ApplyAggOp Count () ())))) (Let __iruid_5791 (MakeStruct (dp_stats (AggLet __iruid_5792 False (Cast Float64 (GetField DP (Ref g))) (Let __iruid_5793 (MakeStruct (n_def (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_5792))))))) (sum (ApplyAggOp Sum () ( (Ref __iruid_5792))))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_5793)) (Cast Float64 (GetField n_def (Ref __iruid_5793))))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (GetField n_called (Ref __iruid_5790))) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (GetField n_called (Ref __iruid_5790)) (GetField n_not_called (Ref __iruid_5790))) (GetField n_filtered (Ref __iruid_5790))))))) (Ref __iruid_5791)))))) (MakeStruct (locus (GetField locus (Ref va))) (alleles (GetField alleles (Ref va))))) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (GetField sample_qc (Ref sa)))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (GetField sample_qc (Ref sa))) (F64 0.97))) (False))) (Coalesce (ApplySpecial `||` Boolean (ApplySpecial `||` Boolean (ApplySpecial `&&` Boolean (Apply isHomRef Boolean (GetField GT (Ref g))) (ApplyComparisonOp LTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref g)) (I32 1)) (ArrayFold __iruid_5794 __iruid_5795 (ArrayFilter __iruid_5796 (GetField AD (Ref g)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_5796)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_5794) (Ref __iruid_5795))))) (F64 0.1))) (ApplySpecial `&&` Boolean (ApplySpecial `&&` Boolean (Apply isHet Boolean (GetField GT (Ref g))) (ApplyComparisonOp GTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref g)) (I32 1)) (ArrayFold __iruid_5797 __iruid_5798 (ArrayFilter __iruid_5799 (GetField AD (Ref g)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_5799)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_5797) (Ref __iruid_5798))))) (F64 0.25))) (ApplyComparisonOp LTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref g)) (I32 1)) (ArrayFold __iruid_5800 __iruid_5801 (ArrayFilter __iruid_5802 (GetField AD (Ref g)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_5802)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_5800) (Ref __iruid_5801))))) (F64 0.75)))) (ApplySpecial `&&` Boolean (Apply isHomVar Boolean (GetField GT (Ref g))) (ApplyComparisonOp GTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref g)) (I32 1)) (ArrayFold __iruid_5803 __iruid_5804 (ArrayFilter __iruid_5805 (GetField AD (Ref g)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_5805)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_5803) (Ref __iruid_5804))))) (F64 0.9)))) (False))) (InsertFields (Ref va) None (variant_qc (Let __iruid_5807 (MakeStruct (call_stats (ApplyAggOp CallStats ( (ArrayLen (GetField alleles (Ref va)))) ( (GetField GT (Ref g)))))) (MakeStruct (AF (GetField AF (GetField call_stats (Ref __iruid_5807)))) (p_value_hwe (GetField p_value (If (ApplyComparisonOp EQ (ArrayLen (GetField alleles (Ref va))) (I32 2)) (Let __iruid_5809 (GetField call_stats (Ref __iruid_5807)) (Let __iruid_5810 (GetField homozygote_count (Ref __iruid_5809)) (Apply hardy_weinberg_test Struct{het_freq_hwe:Float64,p_value:Float64} (ArrayRef (Ref __iruid_5810) (I32 0)) (ApplyBinaryPrimOp Subtract (ArrayRef (GetField AC (Ref __iruid_5809)) (I32 1)) (ApplyBinaryPrimOp Multiply (I32 2) (ArrayRef (Ref __iruid_5810) (I32 1)))) (ArrayRef (Ref __iruid_5810) (I32 1))))) (NA Struct{het_freq_hwe:Float64,p_value:Float64}))))))))) (ApplySpecial `&&` Boolean (Coalesce (ApplyComparisonOp GT (ArrayRef (GetField AF (GetField variant_qc (Ref va))) (I32 1)) (F64 0.01)) (False)) (Coalesce (ApplyComparisonOp GT (GetField p_value_hwe (GetField variant_qc (Ref va))) (F64 1.0E-6)) (False)))) (InsertFields (Ref sa) None (__uid_41 (Cast Float64 (GetField CaffeineConsumption (GetField pheno (Ref sa))))) (__uid_42 (F64 1.0)))) (InsertFields (Ref g) None (__uid_43 (Cast Float64 (Apply nNonRefAlleles Int32 (GetField GT (Ref g))))))) (SelectFields (__uid_41 __uid_42) (Ref sa))) (MakeStruct (locus (GetField locus (Ref va))) (alleles (GetField alleles (Ref va))))) (SelectFields (__uid_43) (Ref g))))) (InsertFields (Ref row) None (y_transpose_x (ArrayRef (GetField y_transpose_x (Ref row)) (I32 0))) (beta (ArrayRef (GetField beta (Ref row)) (I32 0))) (standard_error (ArrayRef (GetField standard_error (Ref row)) (I32 0))) (t_stat (ArrayRef (GetField t_stat (Ref row)) (I32 0))) (p_value (ArrayRef (GetField p_value (Ref row)) (I32 0))))) 2020-01-07 21:48:23 root: INFO: Timer: Time taken for LowerMatrixToTable -- Verify : 0.175ms, total 236.395ms 2020-01-07 21:48:23 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:48:23 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:48:23 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:48:23 root: INFO: Timer: Time taken for LowerMatrixToTable -- LoweringTransformation : 22.459ms, total 258.921ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for LowerMatrixToTable -- Verify : 1.131ms, total 260.150ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for LowerMatrixToTable : 23.984ms, total 260.202ms, tagged coverage 99.1 2020-01-07 21:48:23 root: INFO: optimize LowerMatrixToTable, post Lowering: before: IR size 620: (RelationalLetTable __iruid_5823 (TableAggregate (TableMapGlobals (TableMapGlobals (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32}]}} False "{\"name\":\"TableNativeZippedReader\",\"pathLeft\":\"data/1kg.mt/rows\",\"pathRight\":\"data/1kg.mt/entries\",\"_specLeft\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AC:Array[Int32],AF:Array[Float64],AN:Int32,BaseQRankSum:Float64,ClippingRankSum:Float64,DP:Int32,DS:Boolean,FS:Float64,HaplotypeScore:Float64,InbreedingCoeff:Float64,MLEAC:Array[Int32],MLEAF:Array[Float64],MQ:Float64,MQ0:Int32,MQRankSum:Float64,QD:Float64,ReadPosRankSum:Float64,set:String}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}},\"_specRight\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[],row:Struct{`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32,GQ:Int32,PL:Array[+Int32]}]}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}}}") (InsertFields (Ref global) None (__cols (GetField rows (TableCollect (TableRead Table{global:Struct{},key:[],row:Struct{s:String}} False "{\"name\":\"TableNativeReader\",\"path\":\"data/1kg.mt/cols\",\"_spec\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[s],row:Struct{s:String}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[284]}}}}")))))) (Let __dictfield (ToDict (ArrayMap __iruid_5812 (GetField rows (TableCollect (TableKeyBy () False (TableKeyBy (Sample) False (TableRead Table{global:Struct{},key:[],row:Struct{Sample:String,CaffeineConsumption:Int32}} False "{\"name\":\"TextTableReader\",\"options\":{\"files\":[\"data/1kg_annotations.txt\"],\"typeMapStr\":{},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"impute\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"forceGZ\":false}}"))))) (MakeTuple (0 1) (SelectFields (Sample) (Ref __iruid_5812)) (SelectFields (CaffeineConsumption) (Ref __iruid_5812))))) (InsertFields (Ref global) None (__cols (ArrayMap __iruid_5811 (GetField __cols (Ref global)) (InsertFields (Ref __iruid_5811) None (__uid_3 (ApplyIR get Struct{CaffeineConsumption:Int32} (Ref __dictfield) (MakeStruct (Sample (GetField s (Ref __iruid_5811)))))))))))) (AggLet va False (SelectFields (locus alleles) (Ref row)) (AggArrayPerElement __element_idx __result_idx False True (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let sa (ArrayRef (GetField __cols (Ref global)) (Ref __result_idx)) (AggLet sa False (ArrayRef (GetField __cols (Ref global)) (Ref __element_idx)) (AggLet g False (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __element_idx)) (AggFilter False (ApplyUnaryPrimOp Bang (IsNA (Ref g))) (MakeStruct (__iruid_5816 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (GetField GT (Ref g)))))))) (__iruid_5817 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (IsNA (GetField GT (Ref g))))))) (__iruid_5818 (ApplyAggOp Count () ())) (__iruid_5821 (AggLet __iruid_5792 False (Cast Float64 (GetField DP (Ref g))) (MakeStruct (__iruid_5819 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_5792))))))) (__iruid_5820 (ApplyAggOp Sum () ( (Ref __iruid_5792)))))))))))) (ArrayLen (GetField __cols (Ref global)))))) (TableMapRows (TableToTableApply "{\"name\":\"WrappedMatrixToTableFunction\",\"function\":{\"name\":\"LinearRegressionRowsSingle\",\"yFields\":[\"__y_0\"],\"xField\":\"__uid_40\",\"covFields\":[\"__cov0\"],\"rowBlockSize\":16,\"passThrough\":[]},\"colsFieldName\":\"__cols\",\"entriesFieldName\":\"the entries! [877f12a8827e18f61222c6c8c5fb04a8]\",\"colKey\":[]}" (TableMapRows (TableMapGlobals (TableRename () () () () (TableMapRows (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableFilter (TableMapRows (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableMapGlobals (TableMapGlobals (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32}]}} False "{\"name\":\"TableNativeZippedReader\",\"pathLeft\":\"data/1kg.mt/rows\",\"pathRight\":\"data/1kg.mt/entries\",\"_specLeft\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AC:Array[Int32],AF:Array[Float64],AN:Int32,BaseQRankSum:Float64,ClippingRankSum:Float64,DP:Int32,DS:Boolean,FS:Float64,HaplotypeScore:Float64,InbreedingCoeff:Float64,MLEAC:Array[Int32],MLEAF:Array[Float64],MQ:Float64,MQ0:Int32,MQRankSum:Float64,QD:Float64,ReadPosRankSum:Float64,set:String}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}},\"_specRight\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[],row:Struct{`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32,GQ:Int32,PL:Array[+Int32]}]}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}}}") (InsertFields (Ref global) None (__cols (GetField rows (TableCollect (TableRead Table{global:Struct{},key:[],row:Struct{s:String}} False "{\"name\":\"TableNativeReader\",\"path\":\"data/1kg.mt/cols\",\"_spec\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[s],row:Struct{s:String}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[284]}}}}")))))) (Let __dictfield (ToDict (ArrayMap __iruid_5812 (GetField rows (TableCollect (TableKeyBy () False (TableKeyBy (Sample) False (TableRead Table{global:Struct{},key:[],row:Struct{Sample:String,CaffeineConsumption:Int32}} False "{\"name\":\"TextTableReader\",\"options\":{\"files\":[\"data/1kg_annotations.txt\"],\"typeMapStr\":{},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"impute\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"forceGZ\":false}}"))))) (MakeTuple (0 1) (SelectFields (Sample) (Ref __iruid_5812)) (SelectFields (CaffeineConsumption) (Ref __iruid_5812))))) (InsertFields (Ref global) None (__cols (ArrayMap __iruid_5811 (GetField __cols (Ref global)) (InsertFields (Ref __iruid_5811) None (__uid_3 (ApplyIR get Struct{CaffeineConsumption:Int32} (Ref __dictfield) (MakeStruct (Sample (GetField s (Ref __iruid_5811)))))))))))) (InsertFields (Ref global) None (__cols (Let __iruid_5824 (RelationalRef __iruid_5823 Array[Struct{__iruid_5816:Int64,__iruid_5817:Int64,__iruid_5818:Int64,__iruid_5821:Struct{__iruid_5819:Int64,__iruid_5820:Float64}}]) (ArrayMap __iruid_5822 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __cols_array (GetField __cols (Ref global)) (Let sa (ArrayRef (Ref __cols_array) (Ref __iruid_5822)) (Let __iruid_5824 (RelationalRef __iruid_5823 Array[Struct{__iruid_5816:Int64,__iruid_5817:Int64,__iruid_5818:Int64,__iruid_5821:Struct{__iruid_5819:Int64,__iruid_5820:Float64}}]) (Let __iruid_5825 (ArrayRef (Ref __iruid_5824) (Ref __iruid_5822)) (Let __iruid_5821 (GetField __iruid_5821 (Ref __iruid_5825)) (Let __iruid_5818 (GetField __iruid_5818 (Ref __iruid_5825)) (Let __iruid_5817 (GetField __iruid_5817 (Ref __iruid_5825)) (Let __iruid_5816 (GetField __iruid_5816 (Ref __iruid_5825)) (InsertFields (SelectFields () (Ref sa)) ("pheno" "sample_qc") (pheno (GetField __uid_3 (Ref sa))) (sample_qc (Let __iruid_5790 (MakeStruct (n_called (Ref __iruid_5816)) (n_not_called (Ref __iruid_5817)) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (Ref __iruid_5818)))) (Let __iruid_5791 (MakeStruct (dp_stats (Let __iruid_5820 (GetField __iruid_5820 (Ref __iruid_5821)) (Let __iruid_5819 (GetField __iruid_5819 (Ref __iruid_5821)) (Let __iruid_5793 (MakeStruct (n_def (Ref __iruid_5819)) (sum (Ref __iruid_5820))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_5793)) (Cast Float64 (GetField n_def (Ref __iruid_5793)))))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (GetField n_called (Ref __iruid_5790))) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (GetField n_called (Ref __iruid_5790)) (GetField n_not_called (Ref __iruid_5790))) (GetField n_filtered (Ref __iruid_5790))))))) (Ref __iruid_5791)))))))))))))))))) (InsertFields (Let __iruid_5826 (MakeStruct) (MakeStruct (locus (GetField locus (SelectFields (locus alleles) (Ref row)))) (alleles (GetField alleles (SelectFields (locus alleles) (Ref row)))))) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row))))) (InsertFields (Ref global) None (newColIdx (ArrayFilter i (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let sa (ArrayRef (GetField __cols (Ref global)) (Ref i)) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (GetField sample_qc (Ref sa)))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (GetField sample_qc (Ref sa))) (F64 0.97))) (False))))))) (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap i (GetField newColIdx (Ref global)) (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref i)))))) (SelectFields (__cols) (InsertFields (Ref global) None (__cols (ArrayMap i (GetField newColIdx (Ref global)) (ArrayRef (GetField __cols (Ref global)) (Ref i))))))) (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap i (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let g (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref i)) (If (Let sa (ArrayRef (GetField __cols (Ref global)) (Ref i)) (ApplyUnaryPrimOp Bang (Coalesce (ApplySpecial `||` Boolean (ApplySpecial `||` Boolean (ApplySpecial `&&` Boolean (Apply isHomRef Boolean (GetField GT (Ref g))) (ApplyComparisonOp LTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref g)) (I32 1)) (ArrayFold __iruid_5794 __iruid_5795 (ArrayFilter __iruid_5796 (GetField AD (Ref g)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_5796)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_5794) (Ref __iruid_5795))))) (F64 0.1))) (ApplySpecial `&&` Boolean (ApplySpecial `&&` Boolean (Apply isHet Boolean (GetField GT (Ref g))) (ApplyComparisonOp GTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref g)) (I32 1)) (ArrayFold __iruid_5797 __iruid_5798 (ArrayFilter __iruid_5799 (GetField AD (Ref g)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_5799)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_5797) (Ref __iruid_5798))))) (F64 0.25))) (ApplyComparisonOp LTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref g)) (I32 1)) (ArrayFold __iruid_5800 __iruid_5801 (ArrayFilter __iruid_5802 (GetField AD (Ref g)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_5802)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_5800) (Ref __iruid_5801))))) (F64 0.75)))) (ApplySpecial `&&` Boolean (Apply isHomVar Boolean (GetField GT (Ref g))) (ApplyComparisonOp GTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref g)) (I32 1)) (ArrayFold __iruid_5803 __iruid_5804 (ArrayFilter __iruid_5805 (GetField AD (Ref g)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_5805)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_5803) (Ref __iruid_5804))))) (F64 0.9)))) (False)))) (NA Struct{GT:Call,AD:Array[+Int32],DP:Int32}) (Ref g))))))) (InsertFields (Let __iruid_5827 (MakeStruct) (ArrayAgg i (ArrayFilter i (ArrayRange (I32 0) (ArrayLen (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row))) (I32 1)) (ApplyUnaryPrimOp Bang (IsNA (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref i))))) (AggLet sa False (ArrayRef (GetField __cols (Ref global)) (Ref i)) (AggLet g False (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref i)) (InsertFields (SelectFields (locus alleles) (Ref row)) None (variant_qc (Let __iruid_5807 (MakeStruct (call_stats (ApplyAggOp CallStats ( (ArrayLen (GetField alleles (SelectFields (locus alleles) (Ref row))))) ( (GetField GT (Ref g)))))) (MakeStruct (AF (GetField AF (GetField call_stats (Ref __iruid_5807)))) (p_value_hwe (GetField p_value (If (ApplyComparisonOp EQ (ArrayLen (GetField alleles (SelectFields (locus alleles) (Ref row)))) (I32 2)) (Let __iruid_5809 (GetField call_stats (Ref __iruid_5807)) (Let __iruid_5810 (GetField homozygote_count (Ref __iruid_5809)) (Apply hardy_weinberg_test Struct{het_freq_hwe:Float64,p_value:Float64} (ArrayRef (Ref __iruid_5810) (I32 0)) (ApplyBinaryPrimOp Subtract (ArrayRef (GetField AC (Ref __iruid_5809)) (I32 1)) (ApplyBinaryPrimOp Multiply (I32 2) (ArrayRef (Ref __iruid_5810) (I32 1)))) (ArrayRef (Ref __iruid_5810) (I32 1))))) (NA Struct{het_freq_hwe:Float64,p_value:Float64})))))))))))) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row))))) (ApplySpecial `&&` Boolean (Coalesce (ApplyComparisonOp GT (ArrayRef (GetField AF (GetField variant_qc (SelectFields (locus alleles variant_qc) (Ref row)))) (I32 1)) (F64 0.01)) (False)) (Coalesce (ApplyComparisonOp GT (GetField p_value_hwe (GetField variant_qc (SelectFields (locus alleles variant_qc) (Ref row)))) (F64 1.0E-6)) (False)))) (InsertFields (Ref global) None (__cols (ArrayMap __iruid_5828 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __cols_array (GetField __cols (Ref global)) (Let sa (ArrayRef (Ref __cols_array) (Ref __iruid_5828)) (InsertFields (Ref sa) None (__uid_41 (Cast Float64 (GetField CaffeineConsumption (GetField pheno (Ref sa))))) (__uid_42 (F64 1.0))))))))) (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap i (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let g (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref i)) (Let sa (ArrayRef (GetField __cols (Ref global)) (Ref i)) (InsertFields (Ref g) None (__uid_43 (Cast Float64 (Apply nNonRefAlleles Int32 (GetField GT (Ref g)))))))))))) (InsertFields (Ref global) None (__cols (ArrayMap __iruid_5829 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __cols_array (GetField __cols (Ref global)) (Let sa (ArrayRef (Ref __cols_array) (Ref __iruid_5829)) (SelectFields (__uid_41 __uid_42) (Ref sa)))))))) (InsertFields (Let __iruid_5830 (MakeStruct) (MakeStruct (locus (GetField locus (SelectFields (locus alleles variant_qc) (Ref row)))) (alleles (GetField alleles (SelectFields (locus alleles variant_qc) (Ref row)))))) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row))))) (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap i (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let g (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref i)) (Let sa (ArrayRef (GetField __cols (Ref global)) (Ref i)) (SelectFields (__uid_43) (Ref g))))))))) (CastRename Struct{__cols:Array[Struct{__y_0:Float64,__cov0:Float64}]} (Ref global))) (CastRename Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{__uid_40:Float64}]} (Ref row)))) (InsertFields (Ref row) None (y_transpose_x (ArrayRef (GetField y_transpose_x (Ref row)) (I32 0))) (beta (ArrayRef (GetField beta (Ref row)) (I32 0))) (standard_error (ArrayRef (GetField standard_error (Ref row)) (I32 0))) (t_stat (ArrayRef (GetField t_stat (Ref row)) (I32 0))) (p_value (ArrayRef (GetField p_value (Ref row)) (I32 0)))))) 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 1.805ms, total 271.030ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 4.781ms, total 275.904ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- Simplify : 38.055ms, total 314.039ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 12.237ms, total 326.376ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 1.256ms, total 327.735ms 2020-01-07 21:48:23 root: INFO: Prune: InsertFields: eliminating field '__uid_3' 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 18.286ms, total 346.086ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 1.138ms, total 347.355ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 1.917ms, total 349.337ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- Simplify : 5.048ms, total 354.439ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 12.702ms, total 367.227ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 1.141ms, total 368.470ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 12.506ms, total 381.024ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 1.025ms, total 382.150ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 1.840ms, total 384.037ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- Simplify : 3.073ms, total 387.161ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 10.265ms, total 397.475ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 1.020ms, total 398.592ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 13.449ms, total 412.093ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for Optimize : 142.971ms, total 412.189ms, tagged coverage 99.0 2020-01-07 21:48:23 root: INFO: optimize LowerMatrixToTable, post Lowering: after: IR size 496: (RelationalLetTable __iruid_5823 (TableAggregate (TableKeyBy () False (TableMapGlobals (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,DP:Int32}]}} False "{\"name\":\"TableNativeZippedReader\",\"pathLeft\":\"data/1kg.mt/rows\",\"pathRight\":\"data/1kg.mt/entries\",\"_specLeft\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AC:Array[Int32],AF:Array[Float64],AN:Int32,BaseQRankSum:Float64,ClippingRankSum:Float64,DP:Int32,DS:Boolean,FS:Float64,HaplotypeScore:Float64,InbreedingCoeff:Float64,MLEAC:Array[Int32],MLEAF:Array[Float64],MQ:Float64,MQ0:Int32,MQRankSum:Float64,QD:Float64,ReadPosRankSum:Float64,set:String}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}},\"_specRight\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[],row:Struct{`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32,GQ:Int32,PL:Array[+Int32]}]}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}}}") (InsertFields (Ref global) ("__cols") (__cols (GetField rows (TableCollect (TableRead Table{global:Struct{},key:[],row:Struct{}} False "{\"name\":\"TableNativeReader\",\"path\":\"data/1kg.mt/cols\",\"_spec\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[s],row:Struct{s:String}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[284]}}}}"))))))) (AggArrayPerElement __iruid_6055 __iruid_6056 False True (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (AggLet __iruid_6057 False (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6055)) (AggFilter False (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6057))) (MakeStruct (__iruid_5816 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (GetField GT (Ref __iruid_6057)))))))) (__iruid_5817 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (IsNA (GetField GT (Ref __iruid_6057))))))) (__iruid_5818 (ApplyAggOp Count () ())) (__iruid_5821 (AggLet __iruid_6058 False (Cast Float64 (GetField DP (Ref __iruid_6057))) (MakeStruct (__iruid_5819 (ApplyAggOp Sum () ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6058))))))) (__iruid_5820 (ApplyAggOp Sum () ( (Ref __iruid_6058)))))))))) (ArrayLen (GetField __cols (Ref global))))) (TableMapRows (TableToTableApply "{\"name\":\"WrappedMatrixToTableFunction\",\"function\":{\"name\":\"LinearRegressionRowsSingle\",\"yFields\":[\"__y_0\"],\"xField\":\"__uid_40\",\"covFields\":[\"__cov0\"],\"rowBlockSize\":16,\"passThrough\":[]},\"colsFieldName\":\"__cols\",\"entriesFieldName\":\"the entries! [877f12a8827e18f61222c6c8c5fb04a8]\",\"colKey\":[]}" (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableFilter (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32]}]}} False "{\"name\":\"TableNativeZippedReader\",\"pathLeft\":\"data/1kg.mt/rows\",\"pathRight\":\"data/1kg.mt/entries\",\"_specLeft\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AC:Array[Int32],AF:Array[Float64],AN:Int32,BaseQRankSum:Float64,ClippingRankSum:Float64,DP:Int32,DS:Boolean,FS:Float64,HaplotypeScore:Float64,InbreedingCoeff:Float64,MLEAC:Array[Int32],MLEAF:Array[Float64],MQ:Float64,MQ0:Int32,MQRankSum:Float64,QD:Float64,ReadPosRankSum:Float64,set:String}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}},\"_specRight\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[],row:Struct{`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32,GQ:Int32,PL:Array[+Int32]}]}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}}}") (Let __iruid_6059 (ToDict (ArrayMap __iruid_6060 (GetField rows (TableCollect (TableRead Table{global:Struct{},key:[],row:Struct{Sample:String,CaffeineConsumption:Int32}} False "{\"name\":\"TextTableReader\",\"options\":{\"files\":[\"data/1kg_annotations.txt\"],\"typeMapStr\":{},\"comment\":[],\"separator\":\"\\t\",\"missing\":[\"NA\"],\"hasHeader\":true,\"impute\":true,\"quoteStr\":null,\"skipBlankLines\":false,\"forceBGZ\":false,\"filterAndReplace\":{\"name\":\"TextInputFilterAndReplace\"},\"forceGZ\":false}}"))) (MakeTuple (0 1) (SelectFields (Sample) (Ref __iruid_6060)) (SelectFields (CaffeineConsumption) (Ref __iruid_6060))))) (Let __iruid_6061 (ArrayMap __iruid_6062 (GetField rows (TableCollect (TableRead Table{global:Struct{},key:[],row:Struct{s:String}} False "{\"name\":\"TableNativeReader\",\"path\":\"data/1kg.mt/cols\",\"_spec\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[s],row:Struct{s:String}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[284]}}}}"))) (InsertFields (Ref __iruid_6062) None (__uid_3 (ApplyIR get Struct{CaffeineConsumption:Int32} (Ref __iruid_6059) (MakeStruct (Sample (GetField s (Ref __iruid_6062)))))))) (InsertFields (Ref global) ("__cols") (__cols (ArrayMap __iruid_6063 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_6061)) (I32 1)) (Let __iruid_6064 (ArrayRef (Ref __iruid_6061) (Ref __iruid_6063)) (Let __iruid_6065 (ArrayRef (RelationalRef __iruid_5823 Array[Struct{__iruid_5816:Int64,__iruid_5817:Int64,__iruid_5818:Int64,__iruid_5821:Struct{__iruid_5819:Int64,__iruid_5820:Float64}}]) (Ref __iruid_6063)) (Let __iruid_6066 (GetField __iruid_5821 (Ref __iruid_6065)) (InsertFields (SelectFields () (Ref __iruid_6064)) ("pheno" "sample_qc") (pheno (GetField __uid_3 (Ref __iruid_6064))) (sample_qc (Let __iruid_6067 (MakeStruct (n_called (GetField __iruid_5816 (Ref __iruid_6065))) (n_not_called (GetField __iruid_5817 (Ref __iruid_6065))) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (GetField __iruid_5818 (Ref __iruid_6065))))) (MakeStruct (dp_stats (Let __iruid_6068 (MakeStruct (n_def (GetField __iruid_5819 (Ref __iruid_6066))) (sum (GetField __iruid_5820 (Ref __iruid_6066)))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_6068)) (Cast Float64 (GetField n_def (Ref __iruid_6068)))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (GetField n_called (Ref __iruid_6067))) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (GetField n_called (Ref __iruid_6067)) (GetField n_not_called (Ref __iruid_6067))) (GetField n_filtered (Ref __iruid_6067))))))))))))))))))) (MakeStruct (locus (GetField locus (Ref row))) (alleles (GetField alleles (Ref row))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row))))) (InsertFields (Ref global) None (newColIdx (ArrayFilter __iruid_6069 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6070 (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6069)) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (GetField sample_qc (Ref __iruid_6070)))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (GetField sample_qc (Ref __iruid_6070))) (F64 0.97))) (False))))))) (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap __iruid_6071 (GetField newColIdx (Ref global)) (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6071)))))) (InsertFields (SelectFields (__cols) (Ref global)) ("__cols") (__cols (ArrayMap __iruid_6072 (GetField newColIdx (Ref global)) (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6072)))))) (Let __iruid_6073 (ArrayMap __iruid_6074 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6075 (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6074)) (If (Coalesce (ApplySpecial `||` Boolean (ApplySpecial `||` Boolean (ApplySpecial `&&` Boolean (Apply isHomRef Boolean (GetField GT (Ref __iruid_6075))) (ApplyComparisonOp LTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6075)) (I32 1)) (ArrayFold __iruid_6076 __iruid_6077 (ArrayFilter __iruid_6078 (GetField AD (Ref __iruid_6075)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6078)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6076) (Ref __iruid_6077))))) (F64 0.1))) (ApplySpecial `&&` Boolean (ApplySpecial `&&` Boolean (Apply isHet Boolean (GetField GT (Ref __iruid_6075))) (ApplyComparisonOp GTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6075)) (I32 1)) (ArrayFold __iruid_6079 __iruid_6080 (ArrayFilter __iruid_6081 (GetField AD (Ref __iruid_6075)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6081)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6079) (Ref __iruid_6080))))) (F64 0.25))) (ApplyComparisonOp LTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6075)) (I32 1)) (ArrayFold __iruid_6082 __iruid_6083 (ArrayFilter __iruid_6084 (GetField AD (Ref __iruid_6075)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6084)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6082) (Ref __iruid_6083))))) (F64 0.75)))) (ApplySpecial `&&` Boolean (Apply isHomVar Boolean (GetField GT (Ref __iruid_6075))) (ApplyComparisonOp GTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6075)) (I32 1)) (ArrayFold __iruid_6085 __iruid_6086 (ArrayFilter __iruid_6087 (GetField AD (Ref __iruid_6075)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6087)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6085) (Ref __iruid_6086))))) (F64 0.9)))) (False)) (Ref __iruid_6075) (NA Struct{GT:Call,AD:Array[+Int32]})))) (InsertFields (ArrayAgg __iruid_6088 (ArrayFilter __iruid_6089 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_6073)) (I32 1)) (ApplyUnaryPrimOp Bang (IsNA (ArrayRef (Ref __iruid_6073) (Ref __iruid_6089))))) (InsertFields (SelectFields (locus alleles) (Ref row)) None (variant_qc (Let __iruid_6090 (MakeStruct (call_stats (ApplyAggOp CallStats ( (ArrayLen (GetField alleles (Ref row)))) ( (GetField GT (ArrayRef (Ref __iruid_6073) (Ref __iruid_6088))))))) (MakeStruct (AF (GetField AF (GetField call_stats (Ref __iruid_6090)))) (p_value_hwe (GetField p_value (If (ApplyComparisonOp EQ (ArrayLen (GetField alleles (Ref row))) (I32 2)) (Let __iruid_6091 (GetField call_stats (Ref __iruid_6090)) (Let __iruid_6092 (GetField homozygote_count (Ref __iruid_6091)) (Apply hardy_weinberg_test Struct{het_freq_hwe:Float64,p_value:Float64} (ArrayRef (Ref __iruid_6092) (I32 0)) (ApplyBinaryPrimOp Subtract (ArrayRef (GetField AC (Ref __iruid_6091)) (I32 1)) (ApplyBinaryPrimOp Multiply (I32 2) (ArrayRef (Ref __iruid_6092) (I32 1)))) (ArrayRef (Ref __iruid_6092) (I32 1))))) (NA Struct{het_freq_hwe:Float64,p_value:Float64}))))))))) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref __iruid_6073))))) (ApplySpecial `&&` Boolean (Coalesce (ApplyComparisonOp GT (ArrayRef (GetField AF (GetField variant_qc (Ref row))) (I32 1)) (F64 0.01)) (False)) (Coalesce (ApplyComparisonOp GT (GetField p_value_hwe (GetField variant_qc (Ref row))) (F64 1.0E-6)) (False)))) (InsertFields (Ref global) None (__cols (ArrayMap __iruid_6093 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6094 (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6093)) (InsertFields (Ref __iruid_6094) None (__uid_41 (Cast Float64 (GetField CaffeineConsumption (GetField pheno (Ref __iruid_6094))))) (__uid_42 (F64 1.0)))))))) (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap __iruid_6095 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6096 (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6095)) (InsertFields (Ref __iruid_6096) None (__uid_43 (Cast Float64 (Apply nNonRefAlleles Int32 (GetField GT (Ref __iruid_6096))))))))))) (InsertFields (Ref global) None (__cols (ArrayMap __iruid_6097 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (SelectFields (__uid_41 __uid_42) (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6097))))))) (Let __iruid_6098 (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (MakeStruct (locus (GetField locus (Ref row))) (alleles (GetField alleles (Ref row))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (ArrayMap __iruid_6099 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (SelectFields (__uid_43) (ArrayRef (Ref __iruid_6098) (Ref __iruid_6099)))))))) (CastRename Struct{__cols:Array[Struct{__y_0:Float64,__cov0:Float64}]} (Ref global))) (CastRename Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{__uid_40:Float64}]} (Ref row)))) (InsertFields (Ref row) None (y_transpose_x (ArrayRef (GetField y_transpose_x (Ref row)) (I32 0))) (beta (ArrayRef (GetField beta (Ref row)) (I32 0))) (standard_error (ArrayRef (GetField standard_error (Ref row)) (I32 0))) (t_stat (ArrayRef (GetField t_stat (Ref row)) (I32 0))) (p_value (ArrayRef (GetField p_value (Ref row)) (I32 0)))))) 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- Verify : 0.173ms, total 417.819ms 2020-01-07 21:48:23 root: INFO: interpreting non compilable node: TableCollect 2020-01-07 21:48:23 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:48:23 root: INFO: decoder cache hit 2020-01-07 21:48:23 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:48:23 root: INFO: decoder cache hit 2020-01-07 21:48:23 root: INFO: encoder cache hit 2020-01-07 21:48:23 SparkContext: INFO: Starting job: collect at RVD.scala:676 2020-01-07 21:48:23 DAGScheduler: INFO: Got job 12 (collect at RVD.scala:676) with 1 output partitions 2020-01-07 21:48:23 DAGScheduler: INFO: Final stage: ResultStage 12 (collect at RVD.scala:676) 2020-01-07 21:48:23 DAGScheduler: INFO: Parents of final stage: List() 2020-01-07 21:48:23 DAGScheduler: INFO: Missing parents: List() 2020-01-07 21:48:23 DAGScheduler: INFO: Submitting ResultStage 12 (MapPartitionsRDD[68] at mapPartitions at ContextRDD.scala:217), which has no missing parents 2020-01-07 21:48:23 MemoryStore: INFO: Block broadcast_27 stored as values in memory (estimated size 10.3 KB, free 365.5 MB) 2020-01-07 21:48:23 MemoryStore: INFO: Block broadcast_27_piece0 stored as bytes in memory (estimated size 5.5 KB, free 365.5 MB) 2020-01-07 21:48:23 BlockManagerInfo: INFO: Added broadcast_27_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 5.5 KB, free: 366.2 MB) 2020-01-07 21:48:23 SparkContext: INFO: Created broadcast 27 from broadcast at DAGScheduler.scala:1161 2020-01-07 21:48:23 DAGScheduler: INFO: Submitting 1 missing tasks from ResultStage 12 (MapPartitionsRDD[68] at mapPartitions at ContextRDD.scala:217) (first 15 tasks are for partitions Vector(0)) 2020-01-07 21:48:23 TaskSchedulerImpl: INFO: Adding task set 12.0 with 1 tasks 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 0.0 in stage 12.0 (TID 49, localhost, executor driver, partition 0, PROCESS_LOCAL, 7653 bytes) 2020-01-07 21:48:23 Executor: INFO: Running task 0.0 in stage 12.0 (TID 49) 2020-01-07 21:48:23 Executor: INFO: Finished task 0.0 in stage 12.0 (TID 49). 3897 bytes result sent to driver 2020-01-07 21:48:23 TaskSetManager: INFO: Finished task 0.0 in stage 12.0 (TID 49) in 8 ms on localhost (executor driver) (1/1) 2020-01-07 21:48:23 TaskSchedulerImpl: INFO: Removed TaskSet 12.0, whose tasks have all completed, from pool 2020-01-07 21:48:23 DAGScheduler: INFO: ResultStage 12 (collect at RVD.scala:676) finished in 0.016 s 2020-01-07 21:48:23 DAGScheduler: INFO: Job 12 finished: collect at RVD.scala:676, took 0.018239 s 2020-01-07 21:48:23 root: INFO: decoder cache hit 2020-01-07 21:48:23 root: INFO: took 42.679ms 2020-01-07 21:48:23 root: INFO: interpreting non compilable node: TableAggregate 2020-01-07 21:48:23 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:48:23 root: INFO: decoder cache hit 2020-01-07 21:48:23 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:48:23 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:48:23 root: INFO: decoder cache hit 2020-01-07 21:48:23 root: INFO: decoder cache hit 2020-01-07 21:48:23 root: INFO: decoder cache hit 2020-01-07 21:48:23 root: INFO: decoder cache hit 2020-01-07 21:48:23 root: INFO: optimize Compile: before: IR size 4: (InsertFields (Ref global) ("__cols") (__cols (GetField rows (Literal Struct{rows:Array[Struct{}],global:Struct{}} )))) 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.220ms, total 483.792ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.039ms, total 483.925ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.043ms, total 484.012ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.158ms, total 484.206ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.035ms, total 484.284ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.186ms, total 484.513ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.086ms, total 484.650ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.021ms, total 484.713ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.031ms, total 484.785ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.132ms, total 484.949ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.025ms, total 485.015ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.129ms, total 485.177ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize : 1.665ms, total 485.233ms, tagged coverage 66.4 2020-01-07 21:48:23 root: INFO: optimize Compile: after: IR size 3: (InsertFields (Ref global) ("__cols") (__cols (Literal Array[Struct{}] ))) 2020-01-07 21:48:23 root: INFO: encoder cache hit 2020-01-07 21:48:23 MemoryStore: INFO: Block broadcast_28 stored as values in memory (estimated size 96.0 B, free 365.5 MB) 2020-01-07 21:48:23 MemoryStore: INFO: Block broadcast_28_piece0 stored as bytes in memory (estimated size 65.0 B, free 365.5 MB) 2020-01-07 21:48:23 BlockManagerInfo: INFO: Added broadcast_28_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 65.0 B, free: 366.2 MB) 2020-01-07 21:48:23 SparkContext: INFO: Created broadcast 28 from broadcast at SparkBackend.scala:21 2020-01-07 21:48:23 root: INFO: instruction count: 3: is/hail/codegen/generated/C53. 2020-01-07 21:48:23 root: INFO: instruction count: 57: is/hail/codegen/generated/C53.apply 2020-01-07 21:48:23 root: INFO: instruction count: 16: is/hail/codegen/generated/C53.apply 2020-01-07 21:48:23 root: INFO: instruction count: 14: is/hail/codegen/generated/C53.setPartitionIndex 2020-01-07 21:48:23 root: INFO: instruction count: 6: is/hail/codegen/generated/C53.addPartitionRegion 2020-01-07 21:48:23 root: INFO: instruction count: 27: is/hail/codegen/generated/C53.m3_DECODE_o_struct_of_o_array_of_o_struct_of_ENDEND_TO_o_tuple_of_o_array_of_o_tuple_of_ENDEND 2020-01-07 21:48:23 root: INFO: instruction count: 28: is/hail/codegen/generated/C53.m4_read_fields_group_0 2020-01-07 21:48:23 root: INFO: instruction count: 97: is/hail/codegen/generated/C53.m5_INPLACE_DECODE_o_array_of_o_struct_of_END_TO_o_array_of_o_tuple_of_END 2020-01-07 21:48:23 root: INFO: instruction count: 8: is/hail/codegen/generated/C53.m6_INPLACE_DECODE_o_struct_of_END_TO_o_tuple_of_END 2020-01-07 21:48:23 root: INFO: instruction count: 30: is/hail/codegen/generated/C53.addLiterals 2020-01-07 21:48:23 root: INFO: encoder cache hit 2020-01-07 21:48:23 root: INFO: decoder cache hit 2020-01-07 21:48:23 MemoryStore: INFO: Block broadcast_29 stored as values in memory (estimated size 4.4 KB, free 365.5 MB) 2020-01-07 21:48:23 MemoryStore: INFO: Block broadcast_29_piece0 stored as bytes in memory (estimated size 2.1 KB, free 365.5 MB) 2020-01-07 21:48:23 BlockManagerInfo: INFO: Added broadcast_29_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 2.1 KB, free: 366.2 MB) 2020-01-07 21:48:23 SparkContext: INFO: Created broadcast 29 from broadcast at SparkBackend.scala:21 2020-01-07 21:48:23 root: INFO: optimize Compile: before: IR size 59: (Begin (Let __iruid_6111 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Begin (SeqOp2 0 (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))) ( (ArrayLen (Ref __iruid_6111)))) (ArrayFor __iruid_6112 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_6111)) (I32 1)) (Let __iruid_6055 (ArrayRef (Ref __iruid_6111) (Ref __iruid_6112)) (Let __iruid_6057 (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6055)) (SeqOp2 0 (AggElements () (Int32 +Void) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))) ( (Ref __iruid_6112) (Begin (If (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6057))) (Let __iruid_6058 (Cast Float64 (GetField DP (Ref __iruid_6057))) (Begin (SeqOp2 0 (Sum () (Int64) None) ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (GetField GT (Ref __iruid_6057))))))) (SeqOp2 1 (Sum () (Int64) None) ( (Apply toInt64 Int64 (IsNA (GetField GT (Ref __iruid_6057)))))) (SeqOp2 2 (Count () () None) ()) (SeqOp2 3 (Sum () (Int64) None) ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6058)))))) (SeqOp2 4 (Sum () (Float64) None) ( (Ref __iruid_6058))))) (Begin))))))))))) 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.355ms, total 502.986ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.160ms, total 503.206ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.425ms, total 503.666ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.111ms, total 504.826ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.164ms, total 505.070ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 1.041ms, total 506.149ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.193ms, total 506.442ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.140ms, total 506.621ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.306ms, total 506.964ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.674ms, total 507.672ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.151ms, total 507.889ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.743ms, total 508.666ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.136ms, total 508.855ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.139ms, total 509.069ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.237ms, total 509.336ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 1.774ms, total 511.139ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.183ms, total 511.389ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.817ms, total 512.241ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize : 9.680ms, total 512.308ms, tagged coverage 90.4 2020-01-07 21:48:23 root: INFO: optimize Compile: after: IR size 56: (Begin (Let __iruid_6123 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Begin (SeqOp2 0 (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))) ( (ArrayLen (Ref __iruid_6123)))) (ArrayFor __iruid_6124 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_6123)) (I32 1)) (Let __iruid_6125 (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (ArrayRef (Ref __iruid_6123) (Ref __iruid_6124))) (SeqOp2 0 (AggElements () (Int32 +Void) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))) ( (Ref __iruid_6124) (Begin (If (IsNA (Ref __iruid_6125)) (Begin) (Let __iruid_6126 (Cast Float64 (GetField DP (Ref __iruid_6125))) (Begin (SeqOp2 0 (Sum () (Int64) None) ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (GetField GT (Ref __iruid_6125))))))) (SeqOp2 1 (Sum () (Int64) None) ( (Apply toInt64 Int64 (IsNA (GetField GT (Ref __iruid_6125)))))) (SeqOp2 2 (Count () () None) ()) (SeqOp2 3 (Sum () (Int64) None) ( (Apply toInt64 Int64 (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6126)))))) (SeqOp2 4 (Sum () (Float64) None) ( (Ref __iruid_6126)))))))))))))) 2020-01-07 21:48:23 root: INFO: instruction count: 3: is/hail/codegen/generated/C54. 2020-01-07 21:48:23 root: INFO: instruction count: 22: is/hail/codegen/generated/C54.apply 2020-01-07 21:48:23 root: INFO: instruction count: 20: is/hail/codegen/generated/C54.apply 2020-01-07 21:48:23 root: INFO: instruction count: 21: is/hail/codegen/generated/C54.newAggState 2020-01-07 21:48:23 root: INFO: instruction count: 17: is/hail/codegen/generated/C54.setAggState 2020-01-07 21:48:23 root: INFO: instruction count: 7: is/hail/codegen/generated/C54.getAggOffset 2020-01-07 21:48:23 root: INFO: instruction count: 7: is/hail/codegen/generated/C54.setNumSerialized 2020-01-07 21:48:23 root: INFO: instruction count: 8: is/hail/codegen/generated/C54.setSerializedAgg 2020-01-07 21:48:23 root: INFO: instruction count: 7: is/hail/codegen/generated/C54.getSerializedAgg 2020-01-07 21:48:23 root: INFO: instruction count: 3: is/hail/codegen/generated/C54.m7_create_states_group0 2020-01-07 21:48:23 root: INFO: instruction count: 19: is/hail/codegen/generated/C54.m8_create_states_group0 2020-01-07 21:48:23 root: INFO: instruction count: 7: is/hail/codegen/generated/C54.m9_new_state_group0 2020-01-07 21:48:23 root: INFO: instruction count: 3: is/hail/codegen/generated/C54.m10_create_states_group0 2020-01-07 21:48:23 root: INFO: instruction count: 19: is/hail/codegen/generated/C54.m11_create_states_group0 2020-01-07 21:48:23 root: INFO: instruction count: 53: is/hail/codegen/generated/C54.m12_load_group0 2020-01-07 21:48:23 root: INFO: instruction count: 36: is/hail/codegen/generated/C54.m13_store_group0 2020-01-07 21:48:23 root: INFO: instruction count: 17: is/hail/codegen/generated/C54.m14_method 2020-01-07 21:48:23 root: INFO: instruction count: 300: is/hail/codegen/generated/C54.m15_method 2020-01-07 21:48:23 root: INFO: instruction count: 272: is/hail/codegen/generated/C54.m16_method 2020-01-07 21:48:23 root: INFO: instruction count: 249: is/hail/codegen/generated/C54.m17_method 2020-01-07 21:48:23 root: INFO: instruction count: 74: is/hail/codegen/generated/C54.m18_method 2020-01-07 21:48:23 root: INFO: instruction count: 209: is/hail/codegen/generated/C54.m19_method 2020-01-07 21:48:23 root: INFO: instruction count: 5: is/hail/codegen/generated/C54.m20_method 2020-01-07 21:48:23 root: INFO: instruction count: 113: is/hail/codegen/generated/C54.m21_load_group0 2020-01-07 21:48:23 root: INFO: instruction count: 113: is/hail/codegen/generated/C54.m22_store_group0 2020-01-07 21:48:23 root: INFO: instruction count: 14: is/hail/codegen/generated/C54.setPartitionIndex 2020-01-07 21:48:23 root: INFO: instruction count: 6: is/hail/codegen/generated/C54.addPartitionRegion 2020-01-07 21:48:23 root: INFO: optimize Compile: before: IR size 28: (Let __iruid_6109 (ResultOp2 0 ( (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))))) (MakeTuple (0) (Let __iruid_6113 (GetTupleElement 0 (Ref __iruid_6109)) (ArrayMap __iruid_6056 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_6113)) (I32 1)) (Let __iruid_6110 (ArrayRef (Ref __iruid_6113) (Ref __iruid_6056)) (MakeStruct (__iruid_5816 (GetTupleElement 0 (Ref __iruid_6110))) (__iruid_5817 (GetTupleElement 1 (Ref __iruid_6110))) (__iruid_5818 (GetTupleElement 2 (Ref __iruid_6110))) (__iruid_5821 (MakeStruct (__iruid_5819 (GetTupleElement 3 (Ref __iruid_6110))) (__iruid_5820 (GetTupleElement 4 (Ref __iruid_6110))))))))))) 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.519ms, total 522.081ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.085ms, total 522.218ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.120ms, total 522.369ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.374ms, total 522.778ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.088ms, total 522.911ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.630ms, total 523.573ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.203ms, total 523.823ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.072ms, total 523.946ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.139ms, total 524.116ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.299ms, total 524.447ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.066ms, total 524.553ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.577ms, total 525.163ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- FoldConstants : 0.095ms, total 525.317ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ExtractIntervalFilters : 0.073ms, total 525.427ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- Simplify : 0.134ms, total 525.591ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardLets : 0.338ms, total 525.964ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- ForwardRelationalLets : 0.073ms, total 526.079ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize -- PruneDeadFields : 0.609ms, total 526.721ms 2020-01-07 21:48:23 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation -- Optimize : 5.206ms, total 526.764ms, tagged coverage 86.3 2020-01-07 21:48:23 root: INFO: optimize Compile: after: IR size 28: (Let __iruid_6135 (ResultOp2 0 ( (AggElementsLengthCheck (Int32 +Void) (Int32) ( (Sum () (Int64) None) (Sum () (Int64) None) (Count () () None) (Sum () (Int64) None) (Sum () (Float64) None))))) (MakeTuple (0) (Let __iruid_6136 (GetTupleElement 0 (Ref __iruid_6135)) (ArrayMap __iruid_6137 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_6136)) (I32 1)) (Let __iruid_6138 (ArrayRef (Ref __iruid_6136) (Ref __iruid_6137)) (MakeStruct (__iruid_5816 (GetTupleElement 0 (Ref __iruid_6138))) (__iruid_5817 (GetTupleElement 1 (Ref __iruid_6138))) (__iruid_5818 (GetTupleElement 2 (Ref __iruid_6138))) (__iruid_5821 (MakeStruct (__iruid_5819 (GetTupleElement 3 (Ref __iruid_6138))) (__iruid_5820 (GetTupleElement 4 (Ref __iruid_6138))))))))))) 2020-01-07 21:48:23 root: INFO: instruction count: 3: is/hail/codegen/generated/C55. 2020-01-07 21:48:23 root: INFO: instruction count: 84: is/hail/codegen/generated/C55.apply 2020-01-07 21:48:23 root: INFO: instruction count: 16: is/hail/codegen/generated/C55.apply 2020-01-07 21:48:23 root: INFO: instruction count: 21: is/hail/codegen/generated/C55.newAggState 2020-01-07 21:48:23 root: INFO: instruction count: 17: is/hail/codegen/generated/C55.setAggState 2020-01-07 21:48:23 root: INFO: instruction count: 7: is/hail/codegen/generated/C55.getAggOffset 2020-01-07 21:48:23 root: INFO: instruction count: 7: is/hail/codegen/generated/C55.setNumSerialized 2020-01-07 21:48:23 root: INFO: instruction count: 8: is/hail/codegen/generated/C55.setSerializedAgg 2020-01-07 21:48:23 root: INFO: instruction count: 7: is/hail/codegen/generated/C55.getSerializedAgg 2020-01-07 21:48:23 root: INFO: instruction count: 3: is/hail/codegen/generated/C55.m7_create_states_group0 2020-01-07 21:48:23 root: INFO: instruction count: 19: is/hail/codegen/generated/C55.m8_create_states_group0 2020-01-07 21:48:23 root: INFO: instruction count: 7: is/hail/codegen/generated/C55.m9_new_state_group0 2020-01-07 21:48:23 root: INFO: instruction count: 3: is/hail/codegen/generated/C55.m10_create_states_group0 2020-01-07 21:48:23 root: INFO: instruction count: 19: is/hail/codegen/generated/C55.m11_create_states_group0 2020-01-07 21:48:23 root: INFO: instruction count: 53: is/hail/codegen/generated/C55.m12_load_group0 2020-01-07 21:48:23 root: INFO: instruction count: 36: is/hail/codegen/generated/C55.m13_store_group0 2020-01-07 21:48:23 root: INFO: instruction count: 113: is/hail/codegen/generated/C55.m14_load_group0 2020-01-07 21:48:23 root: INFO: instruction count: 57: is/hail/codegen/generated/C55.m15_array_nested_result_group0 2020-01-07 21:48:23 root: INFO: instruction count: 137: is/hail/codegen/generated/C55.m16_aggs_result_group0 2020-01-07 21:48:23 root: INFO: instruction count: 36: is/hail/codegen/generated/C55.m17_store_group0 2020-01-07 21:48:23 root: INFO: instruction count: 486: is/hail/codegen/generated/C55.m18_method 2020-01-07 21:48:23 root: INFO: instruction count: 275: is/hail/codegen/generated/C55.m19_method 2020-01-07 21:48:23 root: INFO: instruction count: 14: is/hail/codegen/generated/C55.setPartitionIndex 2020-01-07 21:48:23 root: INFO: instruction count: 6: is/hail/codegen/generated/C55.addPartitionRegion 2020-01-07 21:48:23 root: INFO: Aggregate: useTreeAggregate=true 2020-01-07 21:48:23 root: INFO: Aggregate: commutative=true 2020-01-07 21:48:23 SparkContext: INFO: Starting job: runJob at RVD.scala:605 2020-01-07 21:48:23 DAGScheduler: INFO: Got job 13 (runJob at RVD.scala:605) with 2 output partitions 2020-01-07 21:48:23 DAGScheduler: INFO: Final stage: ResultStage 13 (runJob at RVD.scala:605) 2020-01-07 21:48:23 DAGScheduler: INFO: Parents of final stage: List() 2020-01-07 21:48:23 DAGScheduler: INFO: Missing parents: List() 2020-01-07 21:48:23 DAGScheduler: INFO: Submitting ResultStage 13 (MapPartitionsRDD[73] at mapPartitions at ContextRDD.scala:217), which has no missing parents 2020-01-07 21:48:23 MemoryStore: INFO: Block broadcast_30 stored as values in memory (estimated size 71.7 KB, free 365.5 MB) 2020-01-07 21:48:23 MemoryStore: INFO: Block broadcast_30_piece0 stored as bytes in memory (estimated size 33.7 KB, free 365.4 MB) 2020-01-07 21:48:23 BlockManagerInfo: INFO: Added broadcast_30_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 33.7 KB, free: 366.2 MB) 2020-01-07 21:48:23 SparkContext: INFO: Created broadcast 30 from broadcast at DAGScheduler.scala:1161 2020-01-07 21:48:23 DAGScheduler: INFO: Submitting 2 missing tasks from ResultStage 13 (MapPartitionsRDD[73] at mapPartitions at ContextRDD.scala:217) (first 15 tasks are for partitions Vector(0, 1)) 2020-01-07 21:48:23 TaskSchedulerImpl: INFO: Adding task set 13.0 with 2 tasks 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 0.0 in stage 13.0 (TID 50, localhost, executor driver, partition 0, PROCESS_LOCAL, 8296 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 1.0 in stage 13.0 (TID 51, localhost, executor driver, partition 1, PROCESS_LOCAL, 8296 bytes) 2020-01-07 21:48:23 Executor: INFO: Running task 1.0 in stage 13.0 (TID 51) 2020-01-07 21:48:23 Executor: INFO: Running task 0.0 in stage 13.0 (TID 50) 2020-01-07 21:48:23 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:48:23 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:48:23 root: INFO: Index reader cache queries: 8 2020-01-07 21:48:23 root: INFO: Index reader cache hit rate: 0.625 2020-01-07 21:48:23 root: INFO: Index reader cache queries: 8 2020-01-07 21:48:23 root: INFO: Index reader cache hit rate: 0.625 2020-01-07 21:48:23 Executor: INFO: Finished task 0.0 in stage 13.0 (TID 50). 12135 bytes result sent to driver 2020-01-07 21:48:23 Executor: INFO: Finished task 1.0 in stage 13.0 (TID 51). 12135 bytes result sent to driver 2020-01-07 21:48:23 TaskSetManager: INFO: Finished task 0.0 in stage 13.0 (TID 50) in 224 ms on localhost (executor driver) (1/2) 2020-01-07 21:48:23 TaskSetManager: INFO: Finished task 1.0 in stage 13.0 (TID 51) in 224 ms on localhost (executor driver) (2/2) 2020-01-07 21:48:23 TaskSchedulerImpl: INFO: Removed TaskSet 13.0, whose tasks have all completed, from pool 2020-01-07 21:48:23 DAGScheduler: INFO: ResultStage 13 (runJob at RVD.scala:605) finished in 0.235 s 2020-01-07 21:48:23 DAGScheduler: INFO: Job 13 finished: runJob at RVD.scala:605, took 0.237324 s 2020-01-07 21:48:23 root: INFO: took 321.633ms 2020-01-07 21:48:23 root: INFO: interpreting non compilable node: Literal 2020-01-07 21:48:23 root: INFO: took 0.039ms 2020-01-07 21:48:23 root: INFO: interpreting non compilable node: TableCollect 2020-01-07 21:48:23 MemoryStore: INFO: Block broadcast_31 stored as values in memory (estimated size 237.4 KB, free 365.2 MB) 2020-01-07 21:48:23 MemoryStore: INFO: Block broadcast_31_piece0 stored as bytes in memory (estimated size 23.0 KB, free 365.2 MB) 2020-01-07 21:48:23 BlockManagerInfo: INFO: Added broadcast_31_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 23.0 KB, free: 366.1 MB) 2020-01-07 21:48:23 SparkContext: INFO: Created broadcast 31 from textFile at RichSparkContext.scala:16 2020-01-07 21:48:23 FileInputFormat: INFO: Total input paths to process : 1 2020-01-07 21:48:23 root: INFO: encoder cache miss (11 hits, 14 misses, 0.440) 2020-01-07 21:48:23 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_56. 2020-01-07 21:48:23 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_56.apply 2020-01-07 21:48:23 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_56.m0_ENCODE_o_tuple_of_o_stringANDo_int32END_TO_o_struct_of_o_binaryANDo_int32END 2020-01-07 21:48:23 root: INFO: instruction count: 39: is/hail/codegen/generated/C_etypeEncode_56.m1_write_fields_group_0 2020-01-07 21:48:23 root: INFO: instruction count: 16: is/hail/codegen/generated/C_etypeEncode_56.m2_ENCODE_o_binary_TO_o_binary 2020-01-07 21:48:23 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_56.m3_ENCODE_o_int32_TO_o_int32 2020-01-07 21:48:23 SparkContext: INFO: Starting job: collect at RVD.scala:676 2020-01-07 21:48:23 DAGScheduler: INFO: Got job 14 (collect at RVD.scala:676) with 32 output partitions 2020-01-07 21:48:23 DAGScheduler: INFO: Final stage: ResultStage 14 (collect at RVD.scala:676) 2020-01-07 21:48:23 DAGScheduler: INFO: Parents of final stage: List() 2020-01-07 21:48:23 DAGScheduler: INFO: Missing parents: List() 2020-01-07 21:48:23 DAGScheduler: INFO: Submitting ResultStage 14 (MapPartitionsRDD[83] at mapPartitions at ContextRDD.scala:217), which has no missing parents 2020-01-07 21:48:23 MemoryStore: INFO: Block broadcast_32 stored as values in memory (estimated size 16.5 KB, free 365.2 MB) 2020-01-07 21:48:23 MemoryStore: INFO: Block broadcast_32_piece0 stored as bytes in memory (estimated size 8.2 KB, free 365.2 MB) 2020-01-07 21:48:23 BlockManagerInfo: INFO: Added broadcast_32_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 8.2 KB, free: 366.1 MB) 2020-01-07 21:48:23 SparkContext: INFO: Created broadcast 32 from broadcast at DAGScheduler.scala:1161 2020-01-07 21:48:23 DAGScheduler: INFO: Submitting 32 missing tasks from ResultStage 14 (MapPartitionsRDD[83] at mapPartitions at ContextRDD.scala:217) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)) 2020-01-07 21:48:23 TaskSchedulerImpl: INFO: Adding task set 14.0 with 32 tasks 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 0.0 in stage 14.0 (TID 52, localhost, executor driver, partition 0, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 1.0 in stage 14.0 (TID 53, localhost, executor driver, partition 1, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 2.0 in stage 14.0 (TID 54, localhost, executor driver, partition 2, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 3.0 in stage 14.0 (TID 55, localhost, executor driver, partition 3, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 4.0 in stage 14.0 (TID 56, localhost, executor driver, partition 4, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 5.0 in stage 14.0 (TID 57, localhost, executor driver, partition 5, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 6.0 in stage 14.0 (TID 58, localhost, executor driver, partition 6, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 7.0 in stage 14.0 (TID 59, localhost, executor driver, partition 7, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 8.0 in stage 14.0 (TID 60, localhost, executor driver, partition 8, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 9.0 in stage 14.0 (TID 61, localhost, executor driver, partition 9, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 10.0 in stage 14.0 (TID 62, localhost, executor driver, partition 10, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 11.0 in stage 14.0 (TID 63, localhost, executor driver, partition 11, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 12.0 in stage 14.0 (TID 64, localhost, executor driver, partition 12, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 13.0 in stage 14.0 (TID 65, localhost, executor driver, partition 13, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 14.0 in stage 14.0 (TID 66, localhost, executor driver, partition 14, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 15.0 in stage 14.0 (TID 67, localhost, executor driver, partition 15, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 16.0 in stage 14.0 (TID 68, localhost, executor driver, partition 16, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 17.0 in stage 14.0 (TID 69, localhost, executor driver, partition 17, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 18.0 in stage 14.0 (TID 70, localhost, executor driver, partition 18, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 19.0 in stage 14.0 (TID 71, localhost, executor driver, partition 19, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 20.0 in stage 14.0 (TID 72, localhost, executor driver, partition 20, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 21.0 in stage 14.0 (TID 73, localhost, executor driver, partition 21, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 22.0 in stage 14.0 (TID 74, localhost, executor driver, partition 22, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 23.0 in stage 14.0 (TID 75, localhost, executor driver, partition 23, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 24.0 in stage 14.0 (TID 76, localhost, executor driver, partition 24, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 25.0 in stage 14.0 (TID 77, localhost, executor driver, partition 25, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 26.0 in stage 14.0 (TID 78, localhost, executor driver, partition 26, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 27.0 in stage 14.0 (TID 79, localhost, executor driver, partition 27, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 28.0 in stage 14.0 (TID 80, localhost, executor driver, partition 28, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 29.0 in stage 14.0 (TID 81, localhost, executor driver, partition 29, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 30.0 in stage 14.0 (TID 82, localhost, executor driver, partition 30, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 TaskSetManager: INFO: Starting task 31.0 in stage 14.0 (TID 83, localhost, executor driver, partition 31, PROCESS_LOCAL, 7935 bytes) 2020-01-07 21:48:23 Executor: INFO: Running task 1.0 in stage 14.0 (TID 53) 2020-01-07 21:48:23 Executor: INFO: Running task 0.0 in stage 14.0 (TID 52) 2020-01-07 21:48:24 Executor: INFO: Running task 2.0 in stage 14.0 (TID 54) 2020-01-07 21:48:24 Executor: INFO: Running task 3.0 in stage 14.0 (TID 55) 2020-01-07 21:48:24 Executor: INFO: Running task 4.0 in stage 14.0 (TID 56) 2020-01-07 21:48:24 Executor: INFO: Running task 5.0 in stage 14.0 (TID 57) 2020-01-07 21:48:24 Executor: INFO: Running task 6.0 in stage 14.0 (TID 58) 2020-01-07 21:48:24 Executor: INFO: Running task 7.0 in stage 14.0 (TID 59) 2020-01-07 21:48:24 Executor: INFO: Running task 8.0 in stage 14.0 (TID 60) 2020-01-07 21:48:24 Executor: INFO: Running task 10.0 in stage 14.0 (TID 62) 2020-01-07 21:48:24 Executor: INFO: Running task 9.0 in stage 14.0 (TID 61) 2020-01-07 21:48:24 Executor: INFO: Running task 12.0 in stage 14.0 (TID 64) 2020-01-07 21:48:24 Executor: INFO: Running task 11.0 in stage 14.0 (TID 63) 2020-01-07 21:48:24 Executor: INFO: Running task 13.0 in stage 14.0 (TID 65) 2020-01-07 21:48:24 Executor: INFO: Running task 14.0 in stage 14.0 (TID 66) 2020-01-07 21:48:24 Executor: INFO: Running task 15.0 in stage 14.0 (TID 67) 2020-01-07 21:48:24 Executor: INFO: Running task 16.0 in stage 14.0 (TID 68) 2020-01-07 21:48:24 Executor: INFO: Running task 17.0 in stage 14.0 (TID 69) 2020-01-07 21:48:24 Executor: INFO: Running task 18.0 in stage 14.0 (TID 70) 2020-01-07 21:48:24 Executor: INFO: Running task 19.0 in stage 14.0 (TID 71) 2020-01-07 21:48:24 Executor: INFO: Running task 20.0 in stage 14.0 (TID 72) 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:6344+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:22204+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:15860+3172 2020-01-07 21:48:24 Executor: INFO: Running task 22.0 in stage 14.0 (TID 74) 2020-01-07 21:48:24 Executor: INFO: Running task 21.0 in stage 14.0 (TID 73) 2020-01-07 21:48:24 Executor: INFO: Running task 26.0 in stage 14.0 (TID 78) 2020-01-07 21:48:24 Executor: INFO: Running task 27.0 in stage 14.0 (TID 79) 2020-01-07 21:48:24 Executor: INFO: Running task 23.0 in stage 14.0 (TID 75) 2020-01-07 21:48:24 Executor: INFO: Running task 24.0 in stage 14.0 (TID 76) 2020-01-07 21:48:24 Executor: INFO: Running task 28.0 in stage 14.0 (TID 80) 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:0+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:28548+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:38064+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:9516+3172 2020-01-07 21:48:24 Executor: INFO: Running task 30.0 in stage 14.0 (TID 82) 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:41236+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:47580+3172 2020-01-07 21:48:24 Executor: INFO: Running task 29.0 in stage 14.0 (TID 81) 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:25376+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:82472+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:72956+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:31720+3172 2020-01-07 21:48:24 Executor: INFO: Running task 25.0 in stage 14.0 (TID 77) 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:44408+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:60268+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:88816+3172 2020-01-07 21:48:24 Executor: INFO: Running task 31.0 in stage 14.0 (TID 83) 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:53924+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:34892+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:57096+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:76128+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:63440+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:12688+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:69784+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:66612+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:85644+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:3172+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:19032+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:50752+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:95160+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:91988+3172 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:98332+3173 2020-01-07 21:48:24 HadoopRDD: INFO: Input split: file:/fs/home/cliveterence.dar/work/hail/tutorials/data/1kg_annotations.txt:79300+3172 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 280: Executor task launch worker for task 58 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 286: Executor task launch worker for task 64 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 295: Executor task launch worker for task 73 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 294: Executor task launch worker for task 72 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 302: Executor task launch worker for task 80 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 293: Executor task launch worker for task 71 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 281: Executor task launch worker for task 59 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 301: Executor task launch worker for task 79 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 277: Executor task launch worker for task 55 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 288: Executor task launch worker for task 66 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 279: Executor task launch worker for task 57 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 276: Executor task launch worker for task 54 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 291: Executor task launch worker for task 69 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 298: Executor task launch worker for task 76 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 292: Executor task launch worker for task 70 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 285: Executor task launch worker for task 63 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 296: Executor task launch worker for task 74 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 284: Executor task launch worker for task 62 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 287: Executor task launch worker for task 65 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 300: Executor task launch worker for task 78 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 305: Executor task launch worker for task 83 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 290: Executor task launch worker for task 68 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 278: Executor task launch worker for task 56 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 289: Executor task launch worker for task 67 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 283: Executor task launch worker for task 61 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 297: Executor task launch worker for task 75 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 282: Executor task launch worker for task 60 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 304: Executor task launch worker for task 82 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 303: Executor task launch worker for task 81 2020-01-07 21:48:24 root: INFO: RegionPool: initialized for thread 299: Executor task launch worker for task 77 2020-01-07 21:48:24 Executor: INFO: Finished task 28.0 in stage 14.0 (TID 80). 3059 bytes result sent to driver 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 28.0 in stage 14.0 (TID 80) in 42 ms on localhost (executor driver) (1/32) 2020-01-07 21:48:24 Executor: INFO: Finished task 0.0 in stage 14.0 (TID 52). 3035 bytes result sent to driver 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 0.0 in stage 14.0 (TID 52) in 54 ms on localhost (executor driver) (2/32) 2020-01-07 21:48:24 Executor: INFO: Finished task 13.0 in stage 14.0 (TID 65). 3080 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 7.0 in stage 14.0 (TID 59). 3080 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 14.0 in stage 14.0 (TID 66). 3059 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 24.0 in stage 14.0 (TID 76). 3059 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 1.0 in stage 14.0 (TID 53). 3123 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 30.0 in stage 14.0 (TID 82). 3080 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 25.0 in stage 14.0 (TID 77). 3102 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 11.0 in stage 14.0 (TID 63). 3102 bytes result sent to driver 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 14.0 in stage 14.0 (TID 66) in 56 ms on localhost (executor driver) (3/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 7.0 in stage 14.0 (TID 59) in 59 ms on localhost (executor driver) (4/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 24.0 in stage 14.0 (TID 76) in 52 ms on localhost (executor driver) (5/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 13.0 in stage 14.0 (TID 65) in 57 ms on localhost (executor driver) (6/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 1.0 in stage 14.0 (TID 53) in 61 ms on localhost (executor driver) (7/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 30.0 in stage 14.0 (TID 82) in 52 ms on localhost (executor driver) (8/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 11.0 in stage 14.0 (TID 63) in 59 ms on localhost (executor driver) (9/32) 2020-01-07 21:48:24 Executor: INFO: Finished task 23.0 in stage 14.0 (TID 75). 3123 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 19.0 in stage 14.0 (TID 71). 3102 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 6.0 in stage 14.0 (TID 58). 3102 bytes result sent to driver 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 25.0 in stage 14.0 (TID 77) in 54 ms on localhost (executor driver) (10/32) 2020-01-07 21:48:24 Executor: INFO: Finished task 12.0 in stage 14.0 (TID 64). 3059 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 10.0 in stage 14.0 (TID 62). 3080 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 3.0 in stage 14.0 (TID 55). 3102 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 8.0 in stage 14.0 (TID 60). 3059 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 21.0 in stage 14.0 (TID 73). 3059 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 29.0 in stage 14.0 (TID 81). 3123 bytes result sent to driver 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 23.0 in stage 14.0 (TID 75) in 55 ms on localhost (executor driver) (11/32) 2020-01-07 21:48:24 Executor: INFO: Finished task 22.0 in stage 14.0 (TID 74). 3080 bytes result sent to driver 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 19.0 in stage 14.0 (TID 71) in 58 ms on localhost (executor driver) (12/32) 2020-01-07 21:48:24 Executor: INFO: Finished task 20.0 in stage 14.0 (TID 72). 3080 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 4.0 in stage 14.0 (TID 56). 3123 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 18.0 in stage 14.0 (TID 70). 3102 bytes result sent to driver 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 6.0 in stage 14.0 (TID 58) in 63 ms on localhost (executor driver) (13/32) 2020-01-07 21:48:24 Executor: INFO: Finished task 5.0 in stage 14.0 (TID 57). 3102 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 17.0 in stage 14.0 (TID 69). 3080 bytes result sent to driver 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 12.0 in stage 14.0 (TID 64) in 63 ms on localhost (executor driver) (14/32) 2020-01-07 21:48:24 Executor: INFO: Finished task 15.0 in stage 14.0 (TID 67). 3059 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 9.0 in stage 14.0 (TID 61). 3123 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 2.0 in stage 14.0 (TID 54). 3106 bytes result sent to driver 2020-01-07 21:48:24 Executor: INFO: Finished task 27.0 in stage 14.0 (TID 79). 3123 bytes result sent to driver 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 8.0 in stage 14.0 (TID 60) in 65 ms on localhost (executor driver) (15/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 21.0 in stage 14.0 (TID 73) in 61 ms on localhost (executor driver) (16/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 10.0 in stage 14.0 (TID 62) in 65 ms on localhost (executor driver) (17/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 29.0 in stage 14.0 (TID 81) in 57 ms on localhost (executor driver) (18/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 3.0 in stage 14.0 (TID 55) in 67 ms on localhost (executor driver) (19/32) 2020-01-07 21:48:24 Executor: INFO: Finished task 26.0 in stage 14.0 (TID 78). 3059 bytes result sent to driver 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 22.0 in stage 14.0 (TID 74) in 62 ms on localhost (executor driver) (20/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 20.0 in stage 14.0 (TID 72) in 63 ms on localhost (executor driver) (21/32) 2020-01-07 21:48:24 Executor: INFO: Finished task 31.0 in stage 14.0 (TID 83). 3102 bytes result sent to driver 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 4.0 in stage 14.0 (TID 56) in 67 ms on localhost (executor driver) (22/32) 2020-01-07 21:48:24 Executor: INFO: Finished task 16.0 in stage 14.0 (TID 68). 3059 bytes result sent to driver 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 5.0 in stage 14.0 (TID 57) in 68 ms on localhost (executor driver) (23/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 15.0 in stage 14.0 (TID 67) in 65 ms on localhost (executor driver) (24/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 18.0 in stage 14.0 (TID 70) in 64 ms on localhost (executor driver) (25/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 17.0 in stage 14.0 (TID 69) in 64 ms on localhost (executor driver) (26/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 2.0 in stage 14.0 (TID 54) in 69 ms on localhost (executor driver) (27/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 9.0 in stage 14.0 (TID 61) in 68 ms on localhost (executor driver) (28/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 27.0 in stage 14.0 (TID 79) in 61 ms on localhost (executor driver) (29/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 26.0 in stage 14.0 (TID 78) in 61 ms on localhost (executor driver) (30/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 31.0 in stage 14.0 (TID 83) in 60 ms on localhost (executor driver) (31/32) 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 16.0 in stage 14.0 (TID 68) in 67 ms on localhost (executor driver) (32/32) 2020-01-07 21:48:24 TaskSchedulerImpl: INFO: Removed TaskSet 14.0, whose tasks have all completed, from pool 2020-01-07 21:48:24 DAGScheduler: INFO: ResultStage 14 (collect at RVD.scala:676) finished in 0.081 s 2020-01-07 21:48:24 DAGScheduler: INFO: Job 14 finished: collect at RVD.scala:676, took 0.085005 s 2020-01-07 21:48:24 root: INFO: decoder cache miss (34 hits, 10 misses, 0.773 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeDecode_57. 2020-01-07 21:48:24 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeDecode_57.apply 2020-01-07 21:48:24 root: INFO: instruction count: 27: is/hail/codegen/generated/C_etypeDecode_57.m0_DECODE_o_struct_of_o_binaryANDo_int32END_TO_o_tuple_of_o_stringANDo_int32END 2020-01-07 21:48:24 root: INFO: instruction count: 53: is/hail/codegen/generated/C_etypeDecode_57.m1_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 29: is/hail/codegen/generated/C_etypeDecode_57.m2_INPLACE_DECODE_o_binary_TO_o_binary 2020-01-07 21:48:24 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_57.m3_INPLACE_DECODE_o_int32_TO_o_int32 2020-01-07 21:48:24 root: INFO: took 190.370ms 2020-01-07 21:48:24 root: INFO: interpreting non compilable node: TableCollect 2020-01-07 21:48:24 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:48:24 root: INFO: decoder cache hit 2020-01-07 21:48:24 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:48:24 root: INFO: decoder cache miss (35 hits, 11 misses, 0.761 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeDecode_58. 2020-01-07 21:48:24 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeDecode_58.apply 2020-01-07 21:48:24 root: INFO: instruction count: 27: is/hail/codegen/generated/C_etypeDecode_58.m0_DECODE_o_struct_of_o_binaryEND_TO_o_tuple_of_o_stringEND 2020-01-07 21:48:24 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_58.m1_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 29: is/hail/codegen/generated/C_etypeDecode_58.m2_INPLACE_DECODE_o_binary_TO_o_binary 2020-01-07 21:48:24 root: INFO: encoder cache hit 2020-01-07 21:48:24 SparkContext: INFO: Starting job: collect at RVD.scala:676 2020-01-07 21:48:24 DAGScheduler: INFO: Got job 15 (collect at RVD.scala:676) with 1 output partitions 2020-01-07 21:48:24 DAGScheduler: INFO: Final stage: ResultStage 15 (collect at RVD.scala:676) 2020-01-07 21:48:24 DAGScheduler: INFO: Parents of final stage: List() 2020-01-07 21:48:24 DAGScheduler: INFO: Missing parents: List() 2020-01-07 21:48:24 DAGScheduler: INFO: Submitting ResultStage 15 (MapPartitionsRDD[89] at mapPartitions at ContextRDD.scala:217), which has no missing parents 2020-01-07 21:48:24 MemoryStore: INFO: Block broadcast_33 stored as values in memory (estimated size 11.1 KB, free 365.1 MB) 2020-01-07 21:48:24 root: INFO: Index reader cache queries: 8 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 334 2020-01-07 21:48:24 root: INFO: Index reader cache hit rate: 0.625 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 287 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 317 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 303 2020-01-07 21:48:24 root: INFO: Index reader cache queries: 8 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 286 2020-01-07 21:48:24 root: INFO: Index reader cache hit rate: 0.625 2020-01-07 21:48:24 root: INFO: Index reader cache queries: 8 2020-01-07 21:48:24 root: INFO: Index reader cache hit rate: 0.625 2020-01-07 21:48:24 root: INFO: Index reader cache queries: 8 2020-01-07 21:48:24 root: INFO: Index reader cache hit rate: 0.625 2020-01-07 21:48:24 MemoryStore: INFO: Block broadcast_33_piece0 stored as bytes in memory (estimated size 6.0 KB, free 365.1 MB) 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 335 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 285 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 333 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 369 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 319 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 299 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 373 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 331 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 327 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 332 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 352 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 358 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 311 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 349 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 368 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 374 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 289 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 350 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 354 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 300 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 337 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 280 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 330 2020-01-07 21:48:24 BlockManagerInfo: INFO: Added broadcast_33_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 6.0 KB, free: 366.1 MB) 2020-01-07 21:48:24 SparkContext: INFO: Created broadcast 33 from broadcast at DAGScheduler.scala:1161 2020-01-07 21:48:24 BlockManagerInfo: INFO: Removed broadcast_31_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 23.0 KB, free: 366.1 MB) 2020-01-07 21:48:24 DAGScheduler: INFO: Submitting 1 missing tasks from ResultStage 15 (MapPartitionsRDD[89] at mapPartitions at ContextRDD.scala:217) (first 15 tasks are for partitions Vector(0)) 2020-01-07 21:48:24 TaskSchedulerImpl: INFO: Adding task set 15.0 with 1 tasks 2020-01-07 21:48:24 TaskSetManager: INFO: Starting task 0.0 in stage 15.0 (TID 84, localhost, executor driver, partition 0, PROCESS_LOCAL, 7653 bytes) 2020-01-07 21:48:24 Executor: INFO: Running task 0.0 in stage 15.0 (TID 84) 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 343 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 305 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 370 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 297 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 308 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 356 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 346 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 292 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 364 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 290 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 309 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 340 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 301 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 351 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 315 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 316 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 293 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 371 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 288 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 366 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 313 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 357 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 291 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 336 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 279 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 307 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 362 2020-01-07 21:48:24 BlockManagerInfo: INFO: Removed broadcast_26_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 35.1 KB, free: 366.2 MB) 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 283 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 329 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 341 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 375 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 282 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 310 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 353 2020-01-07 21:48:24 BlockManagerInfo: INFO: Removed broadcast_27_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 5.5 KB, free: 366.2 MB) 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 344 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 296 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 276 2020-01-07 21:48:24 BlockManagerInfo: INFO: Removed broadcast_32_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 8.2 KB, free: 366.2 MB) 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 278 2020-01-07 21:48:24 BlockManagerInfo: INFO: Removed broadcast_24_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 7.4 KB, free: 366.2 MB) 2020-01-07 21:48:24 Executor: INFO: Finished task 0.0 in stage 15.0 (TID 84). 6466 bytes result sent to driver 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 277 2020-01-07 21:48:24 TaskSetManager: INFO: Finished task 0.0 in stage 15.0 (TID 84) in 11 ms on localhost (executor driver) (1/1) 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 318 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 372 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 342 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 304 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 320 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 360 2020-01-07 21:48:24 TaskSchedulerImpl: INFO: Removed TaskSet 15.0, whose tasks have all completed, from pool 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 312 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 323 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 314 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 302 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 322 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 361 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 363 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 298 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 359 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 325 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 324 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 347 2020-01-07 21:48:24 DAGScheduler: INFO: ResultStage 15 (collect at RVD.scala:676) finished in 0.068 s 2020-01-07 21:48:24 DAGScheduler: INFO: Job 15 finished: collect at RVD.scala:676, took 0.070442 s 2020-01-07 21:48:24 BlockManagerInfo: INFO: Removed broadcast_25_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 946.0 B, free: 366.2 MB) 2020-01-07 21:48:24 root: INFO: decoder cache hit 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 295 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 367 2020-01-07 21:48:24 root: INFO: took 94.931ms 2020-01-07 21:48:24 BlockManagerInfo: INFO: Removed broadcast_30_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 33.7 KB, free: 366.2 MB) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for InterpretNonCompilable -- LoweringTransformation : 652.398ms, total 1.070s, tagged coverage 2.2 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 321 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 345 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 348 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 284 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 339 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 338 2020-01-07 21:48:24 BlockManagerInfo: INFO: Removed broadcast_29_piece0 on sinin.kps.ku.ac.th:35729 in memory (size: 2.1 KB, free: 366.2 MB) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for InterpretNonCompilable -- Verify : 1.302ms, total 1.072s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for InterpretNonCompilable : 654.159ms, total 1.072s, tagged coverage 2.4 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 326 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 306 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 355 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 294 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 365 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 328 2020-01-07 21:48:24 ContextCleaner: INFO: Cleaned accumulator 281 2020-01-07 21:48:24 root: INFO: optimize InterpretNonCompilable, post Lowering: before: IR size 440: (TableMapRows (TableToTableApply "{\"name\":\"WrappedMatrixToTableFunction\",\"function\":{\"name\":\"LinearRegressionRowsSingle\",\"yFields\":[\"__y_0\"],\"xField\":\"__uid_40\",\"covFields\":[\"__cov0\"],\"rowBlockSize\":16,\"passThrough\":[]},\"colsFieldName\":\"__cols\",\"entriesFieldName\":\"the entries! [877f12a8827e18f61222c6c8c5fb04a8]\",\"colKey\":[]}" (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableFilter (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32]}]}} False "{\"name\":\"TableNativeZippedReader\",\"pathLeft\":\"data/1kg.mt/rows\",\"pathRight\":\"data/1kg.mt/entries\",\"_specLeft\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AC:Array[Int32],AF:Array[Float64],AN:Int32,BaseQRankSum:Float64,ClippingRankSum:Float64,DP:Int32,DS:Boolean,FS:Float64,HaplotypeScore:Float64,InbreedingCoeff:Float64,MLEAC:Array[Int32],MLEAF:Array[Float64],MQ:Float64,MQ0:Int32,MQRankSum:Float64,QD:Float64,ReadPosRankSum:Float64,set:String}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}},\"_specRight\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[],row:Struct{`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32,GQ:Int32,PL:Array[+Int32]}]}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}}}") (Let __iruid_6059 (ToDict (ArrayMap __iruid_6060 (GetField rows (Literal Struct{rows:Array[Struct{Sample:String,CaffeineConsumption:Int32}],global:Struct{}} )) (MakeTuple (0 1) (SelectFields (Sample) (Ref __iruid_6060)) (SelectFields (CaffeineConsumption) (Ref __iruid_6060))))) (Let __iruid_6061 (ArrayMap __iruid_6062 (GetField rows (Literal Struct{rows:Array[Struct{s:String}],global:Struct{}} )) (InsertFields (Ref __iruid_6062) None (__uid_3 (ApplyIR get Struct{CaffeineConsumption:Int32} (Ref __iruid_6059) (MakeStruct (Sample (GetField s (Ref __iruid_6062)))))))) (InsertFields (Ref global) ("__cols") (__cols (ArrayMap __iruid_6063 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_6061)) (I32 1)) (Let __iruid_6064 (ArrayRef (Ref __iruid_6061) (Ref __iruid_6063)) (Let __iruid_6065 (ArrayRef (Literal Array[Struct{__iruid_5816:Int64,__iruid_5817:Int64,__iruid_5818:Int64,__iruid_5821:Struct{__iruid_5819:Int64,__iruid_5820:Float64}}] ) (Ref __iruid_6063)) (Let __iruid_6066 (GetField __iruid_5821 (Ref __iruid_6065)) (InsertFields (SelectFields () (Ref __iruid_6064)) ("pheno" "sample_qc") (pheno (GetField __uid_3 (Ref __iruid_6064))) (sample_qc (Let __iruid_6067 (MakeStruct (n_called (GetField __iruid_5816 (Ref __iruid_6065))) (n_not_called (GetField __iruid_5817 (Ref __iruid_6065))) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (GetField __iruid_5818 (Ref __iruid_6065))))) (MakeStruct (dp_stats (Let __iruid_6068 (MakeStruct (n_def (GetField __iruid_5819 (Ref __iruid_6066))) (sum (GetField __iruid_5820 (Ref __iruid_6066)))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_6068)) (Cast Float64 (GetField n_def (Ref __iruid_6068)))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (GetField n_called (Ref __iruid_6067))) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (GetField n_called (Ref __iruid_6067)) (GetField n_not_called (Ref __iruid_6067))) (GetField n_filtered (Ref __iruid_6067))))))))))))))))))) (MakeStruct (locus (GetField locus (Ref row))) (alleles (GetField alleles (Ref row))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row))))) (InsertFields (Ref global) None (newColIdx (ArrayFilter __iruid_6069 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6070 (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6069)) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (GetField sample_qc (Ref __iruid_6070)))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (GetField sample_qc (Ref __iruid_6070))) (F64 0.97))) (False))))))) (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap __iruid_6071 (GetField newColIdx (Ref global)) (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6071)))))) (InsertFields (SelectFields (__cols) (Ref global)) ("__cols") (__cols (ArrayMap __iruid_6072 (GetField newColIdx (Ref global)) (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6072)))))) (Let __iruid_6073 (ArrayMap __iruid_6074 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6075 (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6074)) (If (Coalesce (ApplySpecial `||` Boolean (ApplySpecial `||` Boolean (ApplySpecial `&&` Boolean (Apply isHomRef Boolean (GetField GT (Ref __iruid_6075))) (ApplyComparisonOp LTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6075)) (I32 1)) (ArrayFold __iruid_6076 __iruid_6077 (ArrayFilter __iruid_6078 (GetField AD (Ref __iruid_6075)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6078)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6076) (Ref __iruid_6077))))) (F64 0.1))) (ApplySpecial `&&` Boolean (ApplySpecial `&&` Boolean (Apply isHet Boolean (GetField GT (Ref __iruid_6075))) (ApplyComparisonOp GTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6075)) (I32 1)) (ArrayFold __iruid_6079 __iruid_6080 (ArrayFilter __iruid_6081 (GetField AD (Ref __iruid_6075)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6081)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6079) (Ref __iruid_6080))))) (F64 0.25))) (ApplyComparisonOp LTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6075)) (I32 1)) (ArrayFold __iruid_6082 __iruid_6083 (ArrayFilter __iruid_6084 (GetField AD (Ref __iruid_6075)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6084)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6082) (Ref __iruid_6083))))) (F64 0.75)))) (ApplySpecial `&&` Boolean (Apply isHomVar Boolean (GetField GT (Ref __iruid_6075))) (ApplyComparisonOp GTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6075)) (I32 1)) (ArrayFold __iruid_6085 __iruid_6086 (ArrayFilter __iruid_6087 (GetField AD (Ref __iruid_6075)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6087)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6085) (Ref __iruid_6086))))) (F64 0.9)))) (False)) (Ref __iruid_6075) (NA Struct{GT:Call,AD:Array[+Int32]})))) (InsertFields (ArrayAgg __iruid_6088 (ArrayFilter __iruid_6089 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_6073)) (I32 1)) (ApplyUnaryPrimOp Bang (IsNA (ArrayRef (Ref __iruid_6073) (Ref __iruid_6089))))) (InsertFields (SelectFields (locus alleles) (Ref row)) None (variant_qc (Let __iruid_6090 (MakeStruct (call_stats (ApplyAggOp CallStats ( (ArrayLen (GetField alleles (Ref row)))) ( (GetField GT (ArrayRef (Ref __iruid_6073) (Ref __iruid_6088))))))) (MakeStruct (AF (GetField AF (GetField call_stats (Ref __iruid_6090)))) (p_value_hwe (GetField p_value (If (ApplyComparisonOp EQ (ArrayLen (GetField alleles (Ref row))) (I32 2)) (Let __iruid_6091 (GetField call_stats (Ref __iruid_6090)) (Let __iruid_6092 (GetField homozygote_count (Ref __iruid_6091)) (Apply hardy_weinberg_test Struct{het_freq_hwe:Float64,p_value:Float64} (ArrayRef (Ref __iruid_6092) (I32 0)) (ApplyBinaryPrimOp Subtract (ArrayRef (GetField AC (Ref __iruid_6091)) (I32 1)) (ApplyBinaryPrimOp Multiply (I32 2) (ArrayRef (Ref __iruid_6092) (I32 1)))) (ArrayRef (Ref __iruid_6092) (I32 1))))) (NA Struct{het_freq_hwe:Float64,p_value:Float64}))))))))) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref __iruid_6073))))) (ApplySpecial `&&` Boolean (Coalesce (ApplyComparisonOp GT (ArrayRef (GetField AF (GetField variant_qc (Ref row))) (I32 1)) (F64 0.01)) (False)) (Coalesce (ApplyComparisonOp GT (GetField p_value_hwe (GetField variant_qc (Ref row))) (F64 1.0E-6)) (False)))) (InsertFields (Ref global) None (__cols (ArrayMap __iruid_6093 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6094 (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6093)) (InsertFields (Ref __iruid_6094) None (__uid_41 (Cast Float64 (GetField CaffeineConsumption (GetField pheno (Ref __iruid_6094))))) (__uid_42 (F64 1.0)))))))) (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap __iruid_6095 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6096 (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6095)) (InsertFields (Ref __iruid_6096) None (__uid_43 (Cast Float64 (Apply nNonRefAlleles Int32 (GetField GT (Ref __iruid_6096))))))))))) (InsertFields (Ref global) None (__cols (ArrayMap __iruid_6097 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (SelectFields (__uid_41 __uid_42) (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6097))))))) (Let __iruid_6098 (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (MakeStruct (locus (GetField locus (Ref row))) (alleles (GetField alleles (Ref row))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (ArrayMap __iruid_6099 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (SelectFields (__uid_43) (ArrayRef (Ref __iruid_6098) (Ref __iruid_6099)))))))) (CastRename Struct{__cols:Array[Struct{__y_0:Float64,__cov0:Float64}]} (Ref global))) (CastRename Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{__uid_40:Float64}]} (Ref row)))) (InsertFields (Ref row) None (y_transpose_x (ArrayRef (GetField y_transpose_x (Ref row)) (I32 0))) (beta (ArrayRef (GetField beta (Ref row)) (I32 0))) (standard_error (ArrayRef (GetField standard_error (Ref row)) (I32 0))) (t_stat (ArrayRef (GetField t_stat (Ref row)) (I32 0))) (p_value (ArrayRef (GetField p_value (Ref row)) (I32 0))))) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 1.734ms, total 1.080s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 4.258ms, total 1.084s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 5.077ms, total 1.089s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 10.939ms, total 1.100s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 1.219ms, total 1.102s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 16.459ms, total 1.118s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 1.085ms, total 1.119s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 2.406ms, total 1.122s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 3.132ms, total 1.125s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 8.558ms, total 1.134s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 1.043ms, total 1.135s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 12.803ms, total 1.148s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 2.714ms, total 1.151s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 6.268ms, total 1.157s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 2.689ms, total 1.160s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 8.624ms, total 1.168s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.860ms, total 1.169s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 10.221ms, total 1.180s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 101.919ms, total 1.180s, tagged coverage 98.2 2020-01-07 21:48:24 root: INFO: optimize InterpretNonCompilable, post Lowering: after: IR size 438: (TableMapRows (TableToTableApply "{\"name\":\"WrappedMatrixToTableFunction\",\"function\":{\"name\":\"LinearRegressionRowsSingle\",\"yFields\":[\"__y_0\"],\"xField\":\"__uid_40\",\"covFields\":[\"__cov0\"],\"rowBlockSize\":16,\"passThrough\":[]},\"colsFieldName\":\"__cols\",\"entriesFieldName\":\"the entries! [877f12a8827e18f61222c6c8c5fb04a8]\",\"colKey\":[]}" (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableFilter (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableMapRows (TableMapGlobals (TableRead Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32]}]}} False "{\"name\":\"TableNativeZippedReader\",\"pathLeft\":\"data/1kg.mt/rows\",\"pathRight\":\"data/1kg.mt/entries\",\"_specLeft\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,qual:Float64,filters:Set[String],info:Struct{AC:Array[Int32],AF:Array[Float64],AN:Int32,BaseQRankSum:Float64,ClippingRankSum:Float64,DP:Int32,DS:Boolean,FS:Float64,HaplotypeScore:Float64,InbreedingCoeff:Float64,MLEAC:Array[Int32],MLEAF:Array[Float64],MQ:Float64,MQ0:Int32,MQRankSum:Float64,QD:Float64,ReadPosRankSum:Float64,set:String}}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}},\"_specRight\":{\"name\":\"TableSpec\",\"file_version\":66304,\"hail_version\":\"0.2.30-2ae07d872f43\",\"references_rel_path\":\"../references\",\"table_type\":\"Table{global:Struct{},key:[],row:Struct{`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32],DP:Int32,GQ:Int32,PL:Array[+Int32]}]}}\",\"components\":{\"globals\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"../globals/rows\"},\"rows\":{\"name\":\"RVDComponentSpec\",\"rel_path\":\"rows\"},\"partition_counts\":{\"name\":\"PartitionCountsComponentSpec\",\"counts\":[5460,5501]}}}}") (Let __iruid_6248 (ToDict (ArrayMap __iruid_6249 (Literal Array[Struct{Sample:String,CaffeineConsumption:Int32}] ) (MakeTuple (0 1) (SelectFields (Sample) (Ref __iruid_6249)) (SelectFields (CaffeineConsumption) (Ref __iruid_6249))))) (Let __iruid_6250 (ArrayMap __iruid_6251 (Literal Array[Struct{s:String}] ) (InsertFields (Ref __iruid_6251) None (__uid_3 (ApplyIR get Struct{CaffeineConsumption:Int32} (Ref __iruid_6248) (MakeStruct (Sample (GetField s (Ref __iruid_6251)))))))) (InsertFields (Ref global) ("__cols") (__cols (ArrayMap __iruid_6252 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_6250)) (I32 1)) (Let __iruid_6253 (ArrayRef (Ref __iruid_6250) (Ref __iruid_6252)) (Let __iruid_6254 (ArrayRef (Literal Array[Struct{__iruid_5816:Int64,__iruid_5817:Int64,__iruid_5818:Int64,__iruid_5821:Struct{__iruid_5819:Int64,__iruid_5820:Float64}}] ) (Ref __iruid_6252)) (Let __iruid_6255 (GetField __iruid_5821 (Ref __iruid_6254)) (InsertFields (SelectFields () (Ref __iruid_6253)) ("pheno" "sample_qc") (pheno (GetField __uid_3 (Ref __iruid_6253))) (sample_qc (Let __iruid_6256 (MakeStruct (n_called (GetField __iruid_5816 (Ref __iruid_6254))) (n_not_called (GetField __iruid_5817 (Ref __iruid_6254))) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (GetField __iruid_5818 (Ref __iruid_6254))))) (MakeStruct (dp_stats (Let __iruid_6257 (MakeStruct (n_def (GetField __iruid_5819 (Ref __iruid_6255))) (sum (GetField __iruid_5820 (Ref __iruid_6255)))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_6257)) (Cast Float64 (GetField n_def (Ref __iruid_6257)))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (GetField n_called (Ref __iruid_6256))) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (GetField n_called (Ref __iruid_6256)) (GetField n_not_called (Ref __iruid_6256))) (GetField n_filtered (Ref __iruid_6256))))))))))))))))))) (MakeStruct (locus (GetField locus (Ref row))) (alleles (GetField alleles (Ref row))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row))))) (InsertFields (Ref global) None (newColIdx (ArrayFilter __iruid_6258 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6259 (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6258)) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (GetField sample_qc (Ref __iruid_6259)))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (GetField sample_qc (Ref __iruid_6259))) (F64 0.97))) (False))))))) (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap __iruid_6260 (GetField newColIdx (Ref global)) (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6260)))))) (InsertFields (SelectFields (__cols) (Ref global)) ("__cols") (__cols (ArrayMap __iruid_6261 (GetField newColIdx (Ref global)) (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6261)))))) (Let __iruid_6262 (ArrayMap __iruid_6263 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6264 (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6263)) (If (Coalesce (ApplySpecial `||` Boolean (ApplySpecial `||` Boolean (ApplySpecial `&&` Boolean (Apply isHomRef Boolean (GetField GT (Ref __iruid_6264))) (ApplyComparisonOp LTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6264)) (I32 1)) (ArrayFold __iruid_6265 __iruid_6266 (ArrayFilter __iruid_6267 (GetField AD (Ref __iruid_6264)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6267)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6265) (Ref __iruid_6266))))) (F64 0.1))) (ApplySpecial `&&` Boolean (ApplySpecial `&&` Boolean (Apply isHet Boolean (GetField GT (Ref __iruid_6264))) (ApplyComparisonOp GTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6264)) (I32 1)) (ArrayFold __iruid_6268 __iruid_6269 (ArrayFilter __iruid_6270 (GetField AD (Ref __iruid_6264)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6270)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6268) (Ref __iruid_6269))))) (F64 0.25))) (ApplyComparisonOp LTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6264)) (I32 1)) (ArrayFold __iruid_6271 __iruid_6272 (ArrayFilter __iruid_6273 (GetField AD (Ref __iruid_6264)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6273)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6271) (Ref __iruid_6272))))) (F64 0.75)))) (ApplySpecial `&&` Boolean (Apply isHomVar Boolean (GetField GT (Ref __iruid_6264))) (ApplyComparisonOp GTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6264)) (I32 1)) (ArrayFold __iruid_6274 __iruid_6275 (ArrayFilter __iruid_6276 (GetField AD (Ref __iruid_6264)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6276)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6274) (Ref __iruid_6275))))) (F64 0.9)))) (False)) (Ref __iruid_6264) (NA Struct{GT:Call,AD:Array[+Int32]})))) (InsertFields (ArrayAgg __iruid_6277 (ArrayFilter __iruid_6278 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_6262)) (I32 1)) (ApplyUnaryPrimOp Bang (IsNA (ArrayRef (Ref __iruid_6262) (Ref __iruid_6278))))) (InsertFields (SelectFields (locus alleles) (Ref row)) None (variant_qc (Let __iruid_6279 (MakeStruct (call_stats (ApplyAggOp CallStats ( (ArrayLen (GetField alleles (Ref row)))) ( (GetField GT (ArrayRef (Ref __iruid_6262) (Ref __iruid_6277))))))) (MakeStruct (AF (GetField AF (GetField call_stats (Ref __iruid_6279)))) (p_value_hwe (GetField p_value (If (ApplyComparisonOp EQ (ArrayLen (GetField alleles (Ref row))) (I32 2)) (Let __iruid_6280 (GetField call_stats (Ref __iruid_6279)) (Let __iruid_6281 (GetField homozygote_count (Ref __iruid_6280)) (Apply hardy_weinberg_test Struct{het_freq_hwe:Float64,p_value:Float64} (ArrayRef (Ref __iruid_6281) (I32 0)) (ApplyBinaryPrimOp Subtract (ArrayRef (GetField AC (Ref __iruid_6280)) (I32 1)) (ApplyBinaryPrimOp Multiply (I32 2) (ArrayRef (Ref __iruid_6281) (I32 1)))) (ArrayRef (Ref __iruid_6281) (I32 1))))) (NA Struct{het_freq_hwe:Float64,p_value:Float64}))))))))) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref __iruid_6262))))) (ApplySpecial `&&` Boolean (Coalesce (ApplyComparisonOp GT (ArrayRef (GetField AF (GetField variant_qc (Ref row))) (I32 1)) (F64 0.01)) (False)) (Coalesce (ApplyComparisonOp GT (GetField p_value_hwe (GetField variant_qc (Ref row))) (F64 1.0E-6)) (False)))) (InsertFields (Ref global) None (__cols (ArrayMap __iruid_6282 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6283 (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6282)) (InsertFields (Ref __iruid_6283) None (__uid_41 (Cast Float64 (GetField CaffeineConsumption (GetField pheno (Ref __iruid_6283))))) (__uid_42 (F64 1.0)))))))) (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap __iruid_6284 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6285 (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6284)) (InsertFields (Ref __iruid_6285) None (__uid_43 (Cast Float64 (Apply nNonRefAlleles Int32 (GetField GT (Ref __iruid_6285))))))))))) (InsertFields (Ref global) None (__cols (ArrayMap __iruid_6286 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (SelectFields (__uid_41 __uid_42) (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6286))))))) (Let __iruid_6287 (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (MakeStruct (locus (GetField locus (Ref row))) (alleles (GetField alleles (Ref row))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (ArrayMap __iruid_6288 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (SelectFields (__uid_43) (ArrayRef (Ref __iruid_6287) (Ref __iruid_6288)))))))) (CastRename Struct{__cols:Array[Struct{__y_0:Float64,__cov0:Float64}]} (Ref global))) (CastRename Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{__uid_40:Float64}]} (Ref row)))) (InsertFields (Ref row) None (y_transpose_x (ArrayRef (GetField y_transpose_x (Ref row)) (I32 0))) (beta (ArrayRef (GetField beta (Ref row)) (I32 0))) (standard_error (ArrayRef (GetField standard_error (Ref row)) (I32 0))) (t_stat (ArrayRef (GetField t_stat (Ref row)) (I32 0))) (p_value (ArrayRef (GetField p_value (Ref row)) (I32 0))))) 2020-01-07 21:48:24 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:48:24 root: INFO: decoder cache hit 2020-01-07 21:48:24 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:48:24 CodecPool: INFO: Got brand-new decompressor [.gz] 2020-01-07 21:48:24 root: INFO: decoder cache hit 2020-01-07 21:48:24 root: INFO: decoder cache hit 2020-01-07 21:48:24 root: INFO: decoder cache hit 2020-01-07 21:48:24 root: INFO: decoder cache hit 2020-01-07 21:48:24 root: INFO: optimize Compile: before: IR size 80: (Let __iruid_6248 (ToDict (ArrayMap __iruid_6249 (Literal Array[Struct{Sample:String,CaffeineConsumption:Int32}] ) (MakeTuple (0 1) (SelectFields (Sample) (Ref __iruid_6249)) (SelectFields (CaffeineConsumption) (Ref __iruid_6249))))) (Let __iruid_6250 (ArrayMap __iruid_6251 (Literal Array[Struct{s:String}] ) (InsertFields (Ref __iruid_6251) None (__uid_3 (ApplyIR get Struct{CaffeineConsumption:Int32} (Ref __iruid_6248) (MakeStruct (Sample (GetField s (Ref __iruid_6251)))))))) (InsertFields (Ref global) ("__cols") (__cols (ArrayMap __iruid_6252 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_6250)) (I32 1)) (Let __iruid_6253 (ArrayRef (Ref __iruid_6250) (Ref __iruid_6252)) (Let __iruid_6254 (ArrayRef (Literal Array[Struct{__iruid_5816:Int64,__iruid_5817:Int64,__iruid_5818:Int64,__iruid_5821:Struct{__iruid_5819:Int64,__iruid_5820:Float64}}] ) (Ref __iruid_6252)) (Let __iruid_6255 (GetField __iruid_5821 (Ref __iruid_6254)) (InsertFields (SelectFields () (Ref __iruid_6253)) ("pheno" "sample_qc") (pheno (GetField __uid_3 (Ref __iruid_6253))) (sample_qc (Let __iruid_6256 (MakeStruct (n_called (GetField __iruid_5816 (Ref __iruid_6254))) (n_not_called (GetField __iruid_5817 (Ref __iruid_6254))) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (GetField __iruid_5818 (Ref __iruid_6254))))) (MakeStruct (dp_stats (Let __iruid_6257 (MakeStruct (n_def (GetField __iruid_5819 (Ref __iruid_6255))) (sum (GetField __iruid_5820 (Ref __iruid_6255)))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_6257)) (Cast Float64 (GetField n_def (Ref __iruid_6257)))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (GetField n_called (Ref __iruid_6256))) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (GetField n_called (Ref __iruid_6256)) (GetField n_not_called (Ref __iruid_6256))) (GetField n_filtered (Ref __iruid_6256)))))))))))))))))) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.142ms, total 1.204s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.135ms, total 1.204s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.520ms, total 1.204s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 1.096ms, total 1.205s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.163ms, total 1.206s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 1.775ms, total 1.207s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.229ms, total 1.208s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.171ms, total 1.208s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.463ms, total 1.208s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 3.199ms, total 1.212s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.176ms, total 1.212s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 1.528ms, total 1.213s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.194ms, total 1.214s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.143ms, total 1.214s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.441ms, total 1.214s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.984ms, total 1.215s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.149ms, total 1.216s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 1.653ms, total 1.217s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 13.991ms, total 1.217s, tagged coverage 94.1 2020-01-07 21:48:24 root: INFO: optimize Compile: after: IR size 80: (Let __iruid_6345 (ToDict (ArrayMap __iruid_6346 (Literal Array[Struct{Sample:String,CaffeineConsumption:Int32}] ) (MakeTuple (0 1) (SelectFields (Sample) (Ref __iruid_6346)) (SelectFields (CaffeineConsumption) (Ref __iruid_6346))))) (Let __iruid_6347 (ArrayMap __iruid_6348 (Literal Array[Struct{s:String}] ) (InsertFields (Ref __iruid_6348) None (__uid_3 (ApplyIR get Struct{CaffeineConsumption:Int32} (Ref __iruid_6345) (MakeStruct (Sample (GetField s (Ref __iruid_6348)))))))) (InsertFields (Ref global) ("__cols") (__cols (ArrayMap __iruid_6349 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_6347)) (I32 1)) (Let __iruid_6350 (ArrayRef (Ref __iruid_6347) (Ref __iruid_6349)) (Let __iruid_6351 (ArrayRef (Literal Array[Struct{__iruid_5816:Int64,__iruid_5817:Int64,__iruid_5818:Int64,__iruid_5821:Struct{__iruid_5819:Int64,__iruid_5820:Float64}}] ) (Ref __iruid_6349)) (Let __iruid_6352 (GetField __iruid_5821 (Ref __iruid_6351)) (InsertFields (SelectFields () (Ref __iruid_6350)) ("pheno" "sample_qc") (pheno (GetField __uid_3 (Ref __iruid_6350))) (sample_qc (Let __iruid_6353 (MakeStruct (n_called (GetField __iruid_5816 (Ref __iruid_6351))) (n_not_called (GetField __iruid_5817 (Ref __iruid_6351))) (n_filtered (ApplyBinaryPrimOp Subtract (I64 10961) (GetField __iruid_5818 (Ref __iruid_6351))))) (MakeStruct (dp_stats (Let __iruid_6354 (MakeStruct (n_def (GetField __iruid_5819 (Ref __iruid_6352))) (sum (GetField __iruid_5820 (Ref __iruid_6352)))) (MakeStruct (mean (ApplyBinaryPrimOp FloatingPointDivide (GetField sum (Ref __iruid_6354)) (Cast Float64 (GetField n_def (Ref __iruid_6354)))))))) (call_rate (ApplyBinaryPrimOp FloatingPointDivide (Cast Float64 (GetField n_called (Ref __iruid_6353))) (Cast Float64 (ApplyBinaryPrimOp Add (ApplyBinaryPrimOp Add (GetField n_called (Ref __iruid_6353)) (GetField n_not_called (Ref __iruid_6353))) (GetField n_filtered (Ref __iruid_6353)))))))))))))))))) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C60. 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C60.apply 2020-01-07 21:48:24 root: INFO: instruction count: 14: is/hail/codegen/generated/C60.apply 2020-01-07 21:48:24 root: INFO: instruction count: 110: is/hail/codegen/generated/C60.m1_method 2020-01-07 21:48:24 root: INFO: instruction count: 33: is/hail/codegen/generated/C60.m2_method 2020-01-07 21:48:24 root: INFO: instruction count: 92: is/hail/codegen/generated/C60.m3_method 2020-01-07 21:48:24 root: INFO: instruction count: 105: is/hail/codegen/generated/C60.m4_method 2020-01-07 21:48:24 root: INFO: encoder cache miss (12 hits, 15 misses, 0.444) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_61. 2020-01-07 21:48:24 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_61.apply 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_61.m0_ENCODE_o_tuple_of_o_array_of_o_tuple_of_o_stringENDANDo_array_of_o_tuple_of_o_stringANDo_int32ENDANDo_array_of_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDENDEND_TO_o_struct_of_o_array_of_o_struct_of_o_binaryENDANDo_array_of_o_struct_of_o_binaryANDo_int32ENDANDo_array_of_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 57: is/hail/codegen/generated/C_etypeEncode_61.m1_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 74: is/hail/codegen/generated/C_etypeEncode_61.m2_ENCODE_o_array_of_o_tuple_of_o_binaryEND_TO_o_array_of_o_struct_of_o_binaryEND 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_61.m3_ENCODE_o_tuple_of_o_binaryEND_TO_o_struct_of_o_binaryEND 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_61.m4_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 16: is/hail/codegen/generated/C_etypeEncode_61.m5_ENCODE_o_binary_TO_o_binary 2020-01-07 21:48:24 root: INFO: instruction count: 74: is/hail/codegen/generated/C_etypeEncode_61.m6_ENCODE_o_array_of_o_tuple_of_o_binaryANDo_int32END_TO_o_array_of_o_struct_of_o_binaryANDo_int32END 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_61.m7_ENCODE_o_tuple_of_o_binaryANDo_int32END_TO_o_struct_of_o_binaryANDo_int32END 2020-01-07 21:48:24 root: INFO: instruction count: 39: is/hail/codegen/generated/C_etypeEncode_61.m8_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_61.m9_ENCODE_o_int32_TO_o_int32 2020-01-07 21:48:24 root: INFO: instruction count: 74: is/hail/codegen/generated/C_etypeEncode_61.m10_ENCODE_o_array_of_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDEND_TO_o_array_of_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_61.m11_ENCODE_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDEND_TO_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 73: is/hail/codegen/generated/C_etypeEncode_61.m12_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_61.m13_ENCODE_o_int64_TO_o_int64 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_61.m14_ENCODE_o_tuple_of_o_int64ANDo_float64END_TO_o_struct_of_o_int64ANDo_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 39: is/hail/codegen/generated/C_etypeEncode_61.m15_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_61.m16_ENCODE_o_float64_TO_o_float64 2020-01-07 21:48:24 MemoryStore: INFO: Block broadcast_34 stored as values in memory (estimated size 70.2 KB, free 365.6 MB) 2020-01-07 21:48:24 MemoryStore: INFO: Block broadcast_34_piece0 stored as bytes in memory (estimated size 23.5 KB, free 365.6 MB) 2020-01-07 21:48:24 BlockManagerInfo: INFO: Added broadcast_34_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 23.5 KB, free: 366.2 MB) 2020-01-07 21:48:24 SparkContext: INFO: Created broadcast 34 from broadcast at SparkBackend.scala:21 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C59. 2020-01-07 21:48:24 root: INFO: instruction count: 54: is/hail/codegen/generated/C59.apply 2020-01-07 21:48:24 root: INFO: instruction count: 16: is/hail/codegen/generated/C59.apply 2020-01-07 21:48:24 root: INFO: instruction count: 1027: is/hail/codegen/generated/C59.m1_method 2020-01-07 21:48:24 root: INFO: instruction count: 99: is/hail/codegen/generated/C59.m2_method 2020-01-07 21:48:24 root: INFO: instruction count: 83: is/hail/codegen/generated/C59.m3_method 2020-01-07 21:48:24 root: INFO: instruction count: 106: is/hail/codegen/generated/C59.m4_method 2020-01-07 21:48:24 root: INFO: instruction count: 248: is/hail/codegen/generated/C59.m5_method 2020-01-07 21:48:24 root: INFO: instruction count: 60: is/hail/codegen/generated/C59.m6_method 2020-01-07 21:48:24 root: INFO: instruction count: 563: is/hail/codegen/generated/C59.m7_method 2020-01-07 21:48:24 root: INFO: instruction count: 87: is/hail/codegen/generated/C59.m8_method 2020-01-07 21:48:24 root: INFO: instruction count: 92: is/hail/codegen/generated/C59.m9_method 2020-01-07 21:48:24 root: INFO: instruction count: 105: is/hail/codegen/generated/C59.m10_method 2020-01-07 21:48:24 root: INFO: instruction count: 37: is/hail/codegen/generated/C59.m11_method 2020-01-07 21:48:24 root: INFO: instruction count: 614: is/hail/codegen/generated/C59.m12_method 2020-01-07 21:48:24 root: INFO: instruction count: 98: is/hail/codegen/generated/C59.m13_method 2020-01-07 21:48:24 root: INFO: instruction count: 43: is/hail/codegen/generated/C59.m14_method 2020-01-07 21:48:24 root: INFO: instruction count: 147: is/hail/codegen/generated/C59.m15_method 2020-01-07 21:48:24 root: INFO: instruction count: 194: is/hail/codegen/generated/C59.m16_method 2020-01-07 21:48:24 root: INFO: instruction count: 128: is/hail/codegen/generated/C59.m17_method 2020-01-07 21:48:24 root: INFO: instruction count: 77: is/hail/codegen/generated/C59.m18_method 2020-01-07 21:48:24 root: INFO: instruction count: 14: is/hail/codegen/generated/C59.setPartitionIndex 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C59.addPartitionRegion 2020-01-07 21:48:24 root: INFO: instruction count: 27: is/hail/codegen/generated/C59.m21_DECODE_o_struct_of_o_array_of_o_struct_of_o_binaryENDANDo_array_of_o_struct_of_o_binaryANDo_int32ENDANDo_array_of_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDENDEND_TO_o_tuple_of_o_array_of_o_tuple_of_o_stringENDANDo_array_of_o_tuple_of_o_stringANDo_int32ENDANDo_array_of_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 78: is/hail/codegen/generated/C59.m22_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 97: is/hail/codegen/generated/C59.m23_INPLACE_DECODE_o_array_of_o_struct_of_o_binaryEND_TO_o_array_of_o_tuple_of_o_binaryEND 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C59.m24_INPLACE_DECODE_o_struct_of_o_binaryEND_TO_o_tuple_of_o_binaryEND 2020-01-07 21:48:24 root: INFO: instruction count: 28: is/hail/codegen/generated/C59.m25_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 29: is/hail/codegen/generated/C59.m26_INPLACE_DECODE_o_binary_TO_o_binary 2020-01-07 21:48:24 root: INFO: instruction count: 97: is/hail/codegen/generated/C59.m27_INPLACE_DECODE_o_array_of_o_struct_of_o_binaryANDo_int32END_TO_o_array_of_o_tuple_of_o_binaryANDo_int32END 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C59.m28_INPLACE_DECODE_o_struct_of_o_binaryANDo_int32END_TO_o_tuple_of_o_binaryANDo_int32END 2020-01-07 21:48:24 root: INFO: instruction count: 53: is/hail/codegen/generated/C59.m29_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 8: is/hail/codegen/generated/C59.m30_INPLACE_DECODE_o_int32_TO_o_int32 2020-01-07 21:48:24 root: INFO: instruction count: 97: is/hail/codegen/generated/C59.m31_INPLACE_DECODE_o_array_of_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDEND_TO_o_array_of_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C59.m32_INPLACE_DECODE_o_struct_of_o_int64ANDo_int64ANDo_int64ANDo_struct_of_o_int64ANDo_float64ENDEND_TO_o_tuple_of_o_int64ANDo_int64ANDo_int64ANDo_tuple_of_o_int64ANDo_float64ENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 103: is/hail/codegen/generated/C59.m33_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 8: is/hail/codegen/generated/C59.m34_INPLACE_DECODE_o_int64_TO_o_int64 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C59.m35_INPLACE_DECODE_o_struct_of_o_int64ANDo_float64END_TO_o_tuple_of_o_int64ANDo_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 53: is/hail/codegen/generated/C59.m36_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 8: is/hail/codegen/generated/C59.m37_INPLACE_DECODE_o_float64_TO_o_float64 2020-01-07 21:48:24 root: INFO: instruction count: 44: is/hail/codegen/generated/C59.addLiterals 2020-01-07 21:48:24 root: INFO: RegionPool: REPORT_THRESHOLD: 518.6K allocated (409.3K blocks / 109.4K chunks), thread 28: Thread-5 2020-01-07 21:48:24 root: INFO: optimize Compile: before: IR size 7: (MakeStruct (locus (GetField locus (Ref row))) (alleles (GetField alleles (Ref row))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)))) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.065ms, total 1.287s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.020ms, total 1.287s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.046ms, total 1.287s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.076ms, total 1.287s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.026ms, total 1.287s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.148ms, total 1.288s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 0.627ms, total 1.288s, tagged coverage 60.9 2020-01-07 21:48:24 root: INFO: optimize Compile: after: IR size 7: (MakeStruct (locus (GetField locus (Ref row))) (alleles (GetField alleles (Ref row))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)))) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C62. 2020-01-07 21:48:24 root: INFO: instruction count: 43: is/hail/codegen/generated/C62.apply 2020-01-07 21:48:24 root: INFO: instruction count: 22: is/hail/codegen/generated/C62.apply 2020-01-07 21:48:24 root: INFO: instruction count: 134: is/hail/codegen/generated/C62.m1_method 2020-01-07 21:48:24 root: INFO: instruction count: 14: is/hail/codegen/generated/C62.setPartitionIndex 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C62.addPartitionRegion 2020-01-07 21:48:24 root: INFO: optimize Compile: before: IR size 28: (InsertFields (Ref global) None (newColIdx (ArrayFilter __iruid_6258 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6259 (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6258)) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (GetField sample_qc (Ref __iruid_6259)))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (GetField sample_qc (Ref __iruid_6259))) (F64 0.97))) (False)))))) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.109ms, total 1.294s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.050ms, total 1.294s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.126ms, total 1.294s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.260ms, total 1.294s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.060ms, total 1.295s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.439ms, total 1.295s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.058ms, total 1.295s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.041ms, total 1.295s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.107ms, total 1.295s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.264ms, total 1.296s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.069ms, total 1.296s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.378ms, total 1.296s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.057ms, total 1.296s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.043ms, total 1.296s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.107ms, total 1.296s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.267ms, total 1.297s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.085ms, total 1.297s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.375ms, total 1.297s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 3.520ms, total 1.297s, tagged coverage 82.2 2020-01-07 21:48:24 root: INFO: optimize Compile: after: IR size 28: (InsertFields (Ref global) None (newColIdx (ArrayFilter __iruid_6372 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6373 (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6372)) (Coalesce (ApplySpecial `&&` Boolean (ApplyComparisonOp GTEQ (GetField mean (GetField dp_stats (GetField sample_qc (Ref __iruid_6373)))) (F64 4.0)) (ApplyComparisonOp GTEQ (GetField call_rate (GetField sample_qc (Ref __iruid_6373))) (F64 0.97))) (False)))))) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C63. 2020-01-07 21:48:24 root: INFO: instruction count: 44: is/hail/codegen/generated/C63.apply 2020-01-07 21:48:24 root: INFO: instruction count: 16: is/hail/codegen/generated/C63.apply 2020-01-07 21:48:24 root: INFO: instruction count: 987: is/hail/codegen/generated/C63.m1_method 2020-01-07 21:48:24 root: INFO: instruction count: 39: is/hail/codegen/generated/C63.m2_method 2020-01-07 21:48:24 root: INFO: instruction count: 14: is/hail/codegen/generated/C63.setPartitionIndex 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C63.addPartitionRegion 2020-01-07 21:48:24 root: INFO: optimize Compile: before: IR size 9: (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap __iruid_6260 (GetField newColIdx (Ref global)) (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6260))))) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.052ms, total 1.304s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.024ms, total 1.304s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.051ms, total 1.304s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.092ms, total 1.304s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.024ms, total 1.304s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.219ms, total 1.305s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.043ms, total 1.305s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.020ms, total 1.305s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.046ms, total 1.305s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.094ms, total 1.305s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.023ms, total 1.305s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.175ms, total 1.305s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.042ms, total 1.305s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.020ms, total 1.305s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.056ms, total 1.305s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.089ms, total 1.306s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.021ms, total 1.306s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.153ms, total 1.306s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 1.813ms, total 1.306s, tagged coverage 68.6 2020-01-07 21:48:24 root: INFO: optimize Compile: after: IR size 9: (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap __iruid_6377 (GetField newColIdx (Ref global)) (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6377))))) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C64. 2020-01-07 21:48:24 root: INFO: instruction count: 46: is/hail/codegen/generated/C64.apply 2020-01-07 21:48:24 root: INFO: instruction count: 22: is/hail/codegen/generated/C64.apply 2020-01-07 21:48:24 root: INFO: instruction count: 437: is/hail/codegen/generated/C64.m1_method 2020-01-07 21:48:24 root: INFO: instruction count: 14: is/hail/codegen/generated/C64.setPartitionIndex 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C64.addPartitionRegion 2020-01-07 21:48:24 root: INFO: encoder cache miss (12 hits, 16 misses, 0.429) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_65. 2020-01-07 21:48:24 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_65.apply 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_65.m0_ENCODE_o_tuple_of_o_array_of_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDENDANDo_array_of_o_int32END_TO_o_struct_of_o_array_of_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDENDANDo_array_of_o_int32END 2020-01-07 21:48:24 root: INFO: instruction count: 39: is/hail/codegen/generated/C_etypeEncode_65.m1_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 74: is/hail/codegen/generated/C_etypeEncode_65.m2_ENCODE_o_array_of_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDEND_TO_o_array_of_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_65.m3_ENCODE_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDEND_TO_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 35: is/hail/codegen/generated/C_etypeEncode_65.m4_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_65.m5_ENCODE_o_tuple_of_o_int32END_TO_o_struct_of_o_int32END 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_65.m6_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_65.m7_ENCODE_o_int32_TO_o_int32 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_65.m8_ENCODE_o_tuple_of_o_tuple_of_o_float64ENDANDo_float64END_TO_o_struct_of_o_struct_of_o_float64ENDANDo_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 37: is/hail/codegen/generated/C_etypeEncode_65.m9_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_65.m10_ENCODE_o_tuple_of_o_float64END_TO_o_struct_of_o_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_65.m11_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_65.m12_ENCODE_o_float64_TO_o_float64 2020-01-07 21:48:24 root: INFO: instruction count: 76: is/hail/codegen/generated/C_etypeEncode_65.m13_ENCODE_o_array_of_o_int32_TO_o_array_of_o_int32 2020-01-07 21:48:24 root: INFO: decoder cache miss (41 hits, 12 misses, 0.774 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeDecode_66. 2020-01-07 21:48:24 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeDecode_66.apply 2020-01-07 21:48:24 root: INFO: instruction count: 27: is/hail/codegen/generated/C_etypeDecode_66.m0_DECODE_o_struct_of_o_array_of_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDENDANDo_array_of_o_int32END_TO_o_tuple_of_o_array_of_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDENDANDo_array_of_o_int32END 2020-01-07 21:48:24 root: INFO: instruction count: 53: is/hail/codegen/generated/C_etypeDecode_66.m1_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 97: is/hail/codegen/generated/C_etypeDecode_66.m2_INPLACE_DECODE_o_array_of_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDEND_TO_o_array_of_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_66.m3_INPLACE_DECODE_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDEND_TO_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 53: is/hail/codegen/generated/C_etypeDecode_66.m4_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_66.m5_INPLACE_DECODE_o_struct_of_o_int32END_TO_o_tuple_of_o_int32END 2020-01-07 21:48:24 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_66.m6_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_66.m7_INPLACE_DECODE_o_int32_TO_o_int32 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_66.m8_INPLACE_DECODE_o_struct_of_o_struct_of_o_float64ENDANDo_float64END_TO_o_tuple_of_o_tuple_of_o_float64ENDANDo_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 53: is/hail/codegen/generated/C_etypeDecode_66.m9_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_66.m10_INPLACE_DECODE_o_struct_of_o_float64END_TO_o_tuple_of_o_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_66.m11_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_66.m12_INPLACE_DECODE_o_float64_TO_o_float64 2020-01-07 21:48:24 root: INFO: instruction count: 97: is/hail/codegen/generated/C_etypeDecode_66.m13_INPLACE_DECODE_o_array_of_o_int32_TO_o_array_of_o_int32 2020-01-07 21:48:24 MemoryStore: INFO: Block broadcast_35 stored as values in memory (estimated size 16.0 KB, free 365.6 MB) 2020-01-07 21:48:24 MemoryStore: INFO: Block broadcast_35_piece0 stored as bytes in memory (estimated size 8.3 KB, free 365.6 MB) 2020-01-07 21:48:24 BlockManagerInfo: INFO: Added broadcast_35_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 8.3 KB, free: 366.2 MB) 2020-01-07 21:48:24 SparkContext: INFO: Created broadcast 35 from broadcast at SparkBackend.scala:21 2020-01-07 21:48:24 root: INFO: optimize Compile: before: IR size 10: (InsertFields (SelectFields (__cols) (Ref global)) ("__cols") (__cols (ArrayMap __iruid_6261 (GetField newColIdx (Ref global)) (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6261))))) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.040ms, total 1.324s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.025ms, total 1.324s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.069ms, total 1.324s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.116ms, total 1.325s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.031ms, total 1.325s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.245ms, total 1.325s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.049ms, total 1.325s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.020ms, total 1.325s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.046ms, total 1.325s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.079ms, total 1.325s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.022ms, total 1.325s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.243ms, total 1.326s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.052ms, total 1.326s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.020ms, total 1.326s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.048ms, total 1.326s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.104ms, total 1.326s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.024ms, total 1.326s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.170ms, total 1.326s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 1.970ms, total 1.326s, tagged coverage 71.2 2020-01-07 21:48:24 root: INFO: optimize Compile: after: IR size 10: (InsertFields (SelectFields (__cols) (Ref global)) ("__cols") (__cols (ArrayMap __iruid_6380 (GetField newColIdx (Ref global)) (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6380))))) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C67. 2020-01-07 21:48:24 root: INFO: instruction count: 93: is/hail/codegen/generated/C67.apply 2020-01-07 21:48:24 root: INFO: instruction count: 16: is/hail/codegen/generated/C67.apply 2020-01-07 21:48:24 root: INFO: instruction count: 374: is/hail/codegen/generated/C67.m1_method 2020-01-07 21:48:24 root: INFO: instruction count: 14: is/hail/codegen/generated/C67.setPartitionIndex 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C67.addPartitionRegion 2020-01-07 21:48:24 root: INFO: optimize Compile: before: IR size 171: (Let __iruid_6262 (ArrayMap __iruid_6263 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6264 (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6263)) (If (Coalesce (ApplySpecial `||` Boolean (ApplySpecial `||` Boolean (ApplySpecial `&&` Boolean (Apply isHomRef Boolean (GetField GT (Ref __iruid_6264))) (ApplyComparisonOp LTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6264)) (I32 1)) (ArrayFold __iruid_6265 __iruid_6266 (ArrayFilter __iruid_6267 (GetField AD (Ref __iruid_6264)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6267)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6265) (Ref __iruid_6266))))) (F64 0.1))) (ApplySpecial `&&` Boolean (ApplySpecial `&&` Boolean (Apply isHet Boolean (GetField GT (Ref __iruid_6264))) (ApplyComparisonOp GTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6264)) (I32 1)) (ArrayFold __iruid_6268 __iruid_6269 (ArrayFilter __iruid_6270 (GetField AD (Ref __iruid_6264)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6270)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6268) (Ref __iruid_6269))))) (F64 0.25))) (ApplyComparisonOp LTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6264)) (I32 1)) (ArrayFold __iruid_6271 __iruid_6272 (ArrayFilter __iruid_6273 (GetField AD (Ref __iruid_6264)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6273)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6271) (Ref __iruid_6272))))) (F64 0.75)))) (ApplySpecial `&&` Boolean (Apply isHomVar Boolean (GetField GT (Ref __iruid_6264))) (ApplyComparisonOp GTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6264)) (I32 1)) (ArrayFold __iruid_6274 __iruid_6275 (ArrayFilter __iruid_6276 (GetField AD (Ref __iruid_6264)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6276)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6274) (Ref __iruid_6275))))) (F64 0.9)))) (False)) (Ref __iruid_6264) (NA Struct{GT:Call,AD:Array[+Int32]})))) (InsertFields (ArrayAgg __iruid_6277 (ArrayFilter __iruid_6278 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_6262)) (I32 1)) (ApplyUnaryPrimOp Bang (IsNA (ArrayRef (Ref __iruid_6262) (Ref __iruid_6278))))) (InsertFields (SelectFields (locus alleles) (Ref row)) None (variant_qc (Let __iruid_6279 (MakeStruct (call_stats (ApplyAggOp CallStats ( (ArrayLen (GetField alleles (Ref row)))) ( (GetField GT (ArrayRef (Ref __iruid_6262) (Ref __iruid_6277))))))) (MakeStruct (AF (GetField AF (GetField call_stats (Ref __iruid_6279)))) (p_value_hwe (GetField p_value (If (ApplyComparisonOp EQ (ArrayLen (GetField alleles (Ref row))) (I32 2)) (Let __iruid_6280 (GetField call_stats (Ref __iruid_6279)) (Let __iruid_6281 (GetField homozygote_count (Ref __iruid_6280)) (Apply hardy_weinberg_test Struct{het_freq_hwe:Float64,p_value:Float64} (ArrayRef (Ref __iruid_6281) (I32 0)) (ApplyBinaryPrimOp Subtract (ArrayRef (GetField AC (Ref __iruid_6280)) (I32 1)) (ApplyBinaryPrimOp Multiply (I32 2) (ArrayRef (Ref __iruid_6281) (I32 1)))) (ArrayRef (Ref __iruid_6281) (I32 1))))) (NA Struct{het_freq_hwe:Float64,p_value:Float64}))))))))) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref __iruid_6262)))) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.476ms, total 1.333s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.219ms, total 1.333s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.692ms, total 1.334s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 6.832ms, total 1.341s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.248ms, total 1.341s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 2.447ms, total 1.344s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.320ms, total 1.344s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.246ms, total 1.345s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.703ms, total 1.345s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 1.139ms, total 1.347s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.247ms, total 1.347s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 2.084ms, total 1.349s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.245ms, total 1.349s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.196ms, total 1.349s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.660ms, total 1.350s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 3.127ms, total 1.353s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.250ms, total 1.354s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 2.050ms, total 1.356s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 23.027ms, total 1.356s, tagged coverage 96.3 2020-01-07 21:48:24 root: INFO: optimize Compile: after: IR size 171: (Let __iruid_6422 (ArrayMap __iruid_6423 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6424 (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6423)) (If (Coalesce (ApplySpecial `||` Boolean (ApplySpecial `||` Boolean (ApplySpecial `&&` Boolean (Apply isHomRef Boolean (GetField GT (Ref __iruid_6424))) (ApplyComparisonOp LTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6424)) (I32 1)) (ArrayFold __iruid_6425 __iruid_6426 (ArrayFilter __iruid_6427 (GetField AD (Ref __iruid_6424)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6427)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6425) (Ref __iruid_6426))))) (F64 0.1))) (ApplySpecial `&&` Boolean (ApplySpecial `&&` Boolean (Apply isHet Boolean (GetField GT (Ref __iruid_6424))) (ApplyComparisonOp GTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6424)) (I32 1)) (ArrayFold __iruid_6428 __iruid_6429 (ArrayFilter __iruid_6430 (GetField AD (Ref __iruid_6424)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6430)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6428) (Ref __iruid_6429))))) (F64 0.25))) (ApplyComparisonOp LTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6424)) (I32 1)) (ArrayFold __iruid_6431 __iruid_6432 (ArrayFilter __iruid_6433 (GetField AD (Ref __iruid_6424)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6433)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6431) (Ref __iruid_6432))))) (F64 0.75)))) (ApplySpecial `&&` Boolean (Apply isHomVar Boolean (GetField GT (Ref __iruid_6424))) (ApplyComparisonOp GTEQ (Cast Float64 (ApplyBinaryPrimOp FloatingPointDivide (ArrayRef (GetField AD (Ref __iruid_6424)) (I32 1)) (ArrayFold __iruid_6434 __iruid_6435 (ArrayFilter __iruid_6436 (GetField AD (Ref __iruid_6424)) (ApplyUnaryPrimOp Bang (IsNA (Ref __iruid_6436)))) (I32 0) (ApplyBinaryPrimOp Add (Ref __iruid_6434) (Ref __iruid_6435))))) (F64 0.9)))) (False)) (Ref __iruid_6424) (NA Struct{GT:Call,AD:Array[+Int32]})))) (InsertFields (ArrayAgg __iruid_6437 (ArrayFilter __iruid_6438 (ArrayRange (I32 0) (ArrayLen (Ref __iruid_6422)) (I32 1)) (ApplyUnaryPrimOp Bang (IsNA (ArrayRef (Ref __iruid_6422) (Ref __iruid_6438))))) (InsertFields (SelectFields (locus alleles) (Ref row)) None (variant_qc (Let __iruid_6439 (MakeStruct (call_stats (ApplyAggOp CallStats ( (ArrayLen (GetField alleles (Ref row)))) ( (GetField GT (ArrayRef (Ref __iruid_6422) (Ref __iruid_6437))))))) (MakeStruct (AF (GetField AF (GetField call_stats (Ref __iruid_6439)))) (p_value_hwe (GetField p_value (If (ApplyComparisonOp EQ (ArrayLen (GetField alleles (Ref row))) (I32 2)) (Let __iruid_6440 (GetField call_stats (Ref __iruid_6439)) (Let __iruid_6441 (GetField homozygote_count (Ref __iruid_6440)) (Apply hardy_weinberg_test Struct{het_freq_hwe:Float64,p_value:Float64} (ArrayRef (Ref __iruid_6441) (I32 0)) (ApplyBinaryPrimOp Subtract (ArrayRef (GetField AC (Ref __iruid_6440)) (I32 1)) (ApplyBinaryPrimOp Multiply (I32 2) (ArrayRef (Ref __iruid_6441) (I32 1)))) (ArrayRef (Ref __iruid_6441) (I32 1))))) (NA Struct{het_freq_hwe:Float64,p_value:Float64}))))))))) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref __iruid_6422)))) 2020-01-07 21:48:24 root: INFO: optimize ArrayAgg/agg.Extract/init: before: IR size 5: (Begin (InitOp2 0 (CallStats (Int32) (Call) None) ( (ArrayLen (GetField alleles (In Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32]}]} 1)))))) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.092ms, total 1.368s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.073ms, total 1.368s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.036ms, total 1.368s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.174ms, total 1.368s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.023ms, total 1.368s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.098ms, total 1.368s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 0.744ms, total 1.368s, tagged coverage 66.8 2020-01-07 21:48:24 root: INFO: optimize ArrayAgg/agg.Extract/init: after: IR size 5: (Begin (InitOp2 0 (CallStats (Int32) (Call) None) ( (ArrayLen (GetField alleles (In Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32]}]} 1)))))) 2020-01-07 21:48:24 root: INFO: optimize ArrayAgg/agg.Extract/perElt: before: IR size 6: (Begin (SeqOp2 0 (CallStats (Int32) (Call) None) ( (GetField GT (ArrayRef (Ref __iruid_6422) (Ref __iruid_6437)))))) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.034ms, total 1.369s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.034ms, total 1.369s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.028ms, total 1.369s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.113ms, total 1.369s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.016ms, total 1.369s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.097ms, total 1.369s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 0.506ms, total 1.369s, tagged coverage 63.6 2020-01-07 21:48:24 root: INFO: optimize ArrayAgg/agg.Extract/perElt: after: IR size 6: (Begin (SeqOp2 0 (CallStats (Int32) (Call) None) ( (GetField GT (ArrayRef (Ref __iruid_6422) (Ref __iruid_6437)))))) 2020-01-07 21:48:24 root: INFO: optimize ArrayAgg/agg.Extract/postAggIR: before: IR size 44: (Let __iruid_6442 (ResultOp2 0 ( (CallStats (Int32) (Call) None))) (InsertFields (SelectFields (locus alleles) (In Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32]}]} 1)) None (variant_qc (Let __iruid_6439 (MakeStruct (call_stats (GetTupleElement 0 (Ref __iruid_6442)))) (MakeStruct (AF (GetField AF (GetField call_stats (Ref __iruid_6439)))) (p_value_hwe (GetField p_value (If (ApplyComparisonOp EQ (ArrayLen (GetField alleles (In Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32]}]} 1))) (I32 2)) (Let __iruid_6440 (GetField call_stats (Ref __iruid_6439)) (Let __iruid_6441 (GetField homozygote_count (Ref __iruid_6440)) (Apply hardy_weinberg_test Struct{het_freq_hwe:Float64,p_value:Float64} (ArrayRef (Ref __iruid_6441) (I32 0)) (ApplyBinaryPrimOp Subtract (ArrayRef (GetField AC (Ref __iruid_6440)) (I32 1)) (ApplyBinaryPrimOp Multiply (I32 2) (ArrayRef (Ref __iruid_6441) (I32 1)))) (ArrayRef (Ref __iruid_6441) (I32 1))))) (NA Struct{het_freq_hwe:Float64,p_value:Float64}))))))))) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.307ms, total 1.370s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.090ms, total 1.370s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.180ms, total 1.370s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.486ms, total 1.371s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.111ms, total 1.371s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.779ms, total 1.372s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.157ms, total 1.372s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.069ms, total 1.372s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.197ms, total 1.372s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.456ms, total 1.373s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.083ms, total 1.373s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 1.361ms, total 1.374s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.136ms, total 1.374s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.066ms, total 1.375s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.182ms, total 1.375s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.387ms, total 1.375s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.082ms, total 1.375s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.606ms, total 1.376s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 6.441ms, total 1.376s, tagged coverage 89.0 2020-01-07 21:48:24 root: INFO: optimize ArrayAgg/agg.Extract/postAggIR: after: IR size 44: (Let __iruid_6451 (ResultOp2 0 ( (CallStats (Int32) (Call) None))) (InsertFields (SelectFields (locus alleles) (In Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32]}]} 1)) None (variant_qc (Let __iruid_6452 (MakeStruct (call_stats (GetTupleElement 0 (Ref __iruid_6451)))) (MakeStruct (AF (GetField AF (GetField call_stats (Ref __iruid_6452)))) (p_value_hwe (GetField p_value (If (ApplyComparisonOp EQ (ArrayLen (GetField alleles (In Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GT:Call,AD:Array[+Int32]}]} 1))) (I32 2)) (Let __iruid_6453 (GetField call_stats (Ref __iruid_6452)) (Let __iruid_6454 (GetField homozygote_count (Ref __iruid_6453)) (Apply hardy_weinberg_test Struct{het_freq_hwe:Float64,p_value:Float64} (ArrayRef (Ref __iruid_6454) (I32 0)) (ApplyBinaryPrimOp Subtract (ArrayRef (GetField AC (Ref __iruid_6453)) (I32 1)) (ApplyBinaryPrimOp Multiply (I32 2) (ArrayRef (Ref __iruid_6454) (I32 1)))) (ArrayRef (Ref __iruid_6454) (I32 1))))) (NA Struct{het_freq_hwe:Float64,p_value:Float64}))))))))) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C68. 2020-01-07 21:48:24 root: INFO: instruction count: 850: is/hail/codegen/generated/C68.apply 2020-01-07 21:48:24 root: INFO: instruction count: 22: is/hail/codegen/generated/C68.apply 2020-01-07 21:48:24 root: INFO: instruction count: 6247: is/hail/codegen/generated/C68.m1_method 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C68.m2_method 2020-01-07 21:48:24 root: INFO: instruction count: 39: is/hail/codegen/generated/C68.m3_method 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C68.m4_method 2020-01-07 21:48:24 root: INFO: instruction count: 39: is/hail/codegen/generated/C68.m5_method 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C68.m6_method 2020-01-07 21:48:24 root: INFO: instruction count: 17: is/hail/codegen/generated/C68.m7_create_states_group0 2020-01-07 21:48:24 root: INFO: instruction count: 7: is/hail/codegen/generated/C68.m8_new_state_group0 2020-01-07 21:48:24 root: INFO: instruction count: 666: is/hail/codegen/generated/C68.m9_method 2020-01-07 21:48:24 root: INFO: instruction count: 457: is/hail/codegen/generated/C68.m10_aggs_result_group0 2020-01-07 21:48:24 root: INFO: instruction count: 36: is/hail/codegen/generated/C68.m11_store_group0 2020-01-07 21:48:24 root: INFO: instruction count: 187: is/hail/codegen/generated/C68.m12_method 2020-01-07 21:48:24 root: INFO: instruction count: 764: is/hail/codegen/generated/C68.m13_method 2020-01-07 21:48:24 root: INFO: instruction count: 37: is/hail/codegen/generated/C68.m14_method 2020-01-07 21:48:24 root: INFO: instruction count: 50: is/hail/codegen/generated/C68.m15_method 2020-01-07 21:48:24 root: INFO: instruction count: 14: is/hail/codegen/generated/C68.setPartitionIndex 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C68.addPartitionRegion 2020-01-07 21:48:24 root: INFO: encoder cache miss (12 hits, 17 misses, 0.414) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_69. 2020-01-07 21:48:24 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_69.apply 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_69.m0_ENCODE_o_tuple_of_o_array_of_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDENDEND_TO_o_struct_of_o_array_of_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_69.m1_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 74: is/hail/codegen/generated/C_etypeEncode_69.m2_ENCODE_o_array_of_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDEND_TO_o_array_of_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_69.m3_ENCODE_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDEND_TO_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 35: is/hail/codegen/generated/C_etypeEncode_69.m4_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_69.m5_ENCODE_o_tuple_of_o_int32END_TO_o_struct_of_o_int32END 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_69.m6_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_69.m7_ENCODE_o_int32_TO_o_int32 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_69.m8_ENCODE_o_tuple_of_o_tuple_of_o_float64ENDANDo_float64END_TO_o_struct_of_o_struct_of_o_float64ENDANDo_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 37: is/hail/codegen/generated/C_etypeEncode_69.m9_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_69.m10_ENCODE_o_tuple_of_o_float64END_TO_o_struct_of_o_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_69.m11_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_69.m12_ENCODE_o_float64_TO_o_float64 2020-01-07 21:48:24 root: INFO: decoder cache miss (41 hits, 13 misses, 0.759 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeDecode_70. 2020-01-07 21:48:24 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeDecode_70.apply 2020-01-07 21:48:24 root: INFO: instruction count: 27: is/hail/codegen/generated/C_etypeDecode_70.m0_DECODE_o_struct_of_o_array_of_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDENDEND_TO_o_tuple_of_o_array_of_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_70.m1_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 97: is/hail/codegen/generated/C_etypeDecode_70.m2_INPLACE_DECODE_o_array_of_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDEND_TO_o_array_of_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_70.m3_INPLACE_DECODE_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDEND_TO_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 53: is/hail/codegen/generated/C_etypeDecode_70.m4_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_70.m5_INPLACE_DECODE_o_struct_of_o_int32END_TO_o_tuple_of_o_int32END 2020-01-07 21:48:24 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_70.m6_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_70.m7_INPLACE_DECODE_o_int32_TO_o_int32 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_70.m8_INPLACE_DECODE_o_struct_of_o_struct_of_o_float64ENDANDo_float64END_TO_o_tuple_of_o_tuple_of_o_float64ENDANDo_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 53: is/hail/codegen/generated/C_etypeDecode_70.m9_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_70.m10_INPLACE_DECODE_o_struct_of_o_float64END_TO_o_tuple_of_o_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_70.m11_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_70.m12_INPLACE_DECODE_o_float64_TO_o_float64 2020-01-07 21:48:24 MemoryStore: INFO: Block broadcast_36 stored as values in memory (estimated size 14.0 KB, free 365.5 MB) 2020-01-07 21:48:24 MemoryStore: INFO: Block broadcast_36_piece0 stored as bytes in memory (estimated size 7.0 KB, free 365.5 MB) 2020-01-07 21:48:24 BlockManagerInfo: INFO: Added broadcast_36_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 7.0 KB, free: 366.2 MB) 2020-01-07 21:48:24 SparkContext: INFO: Created broadcast 36 from broadcast at SparkBackend.scala:21 2020-01-07 21:48:24 root: INFO: optimize Compile: before: IR size 17: (ApplySpecial `&&` Boolean (Coalesce (ApplyComparisonOp GT (ArrayRef (GetField AF (GetField variant_qc (Ref row))) (I32 1)) (F64 0.01)) (False)) (Coalesce (ApplyComparisonOp GT (GetField p_value_hwe (GetField variant_qc (Ref row))) (F64 1.0E-6)) (False))) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.125ms, total 1.420s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.039ms, total 1.420s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.102ms, total 1.420s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.119ms, total 1.420s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.039ms, total 1.420s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.200ms, total 1.420s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 0.881ms, total 1.420s, tagged coverage 70.7 2020-01-07 21:48:24 root: INFO: optimize Compile: after: IR size 17: (ApplySpecial `&&` Boolean (Coalesce (ApplyComparisonOp GT (ArrayRef (GetField AF (GetField variant_qc (Ref row))) (I32 1)) (F64 0.01)) (False)) (Coalesce (ApplyComparisonOp GT (GetField p_value_hwe (GetField variant_qc (Ref row))) (F64 1.0E-6)) (False))) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C71. 2020-01-07 21:48:24 root: INFO: instruction count: 462: is/hail/codegen/generated/C71.apply 2020-01-07 21:48:24 root: INFO: instruction count: 22: is/hail/codegen/generated/C71.apply 2020-01-07 21:48:24 root: INFO: instruction count: 35: is/hail/codegen/generated/C71.m1_method 2020-01-07 21:48:24 root: INFO: instruction count: 14: is/hail/codegen/generated/C71.setPartitionIndex 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C71.addPartitionRegion 2020-01-07 21:48:24 root: INFO: optimize Compile: before: IR size 21: (InsertFields (Ref global) None (__cols (ArrayMap __iruid_6282 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6283 (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6282)) (InsertFields (Ref __iruid_6283) None (__uid_41 (Cast Float64 (GetField CaffeineConsumption (GetField pheno (Ref __iruid_6283))))) (__uid_42 (F64 1.0))))))) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.079ms, total 1.429s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.057ms, total 1.429s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.118ms, total 1.430s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.190ms, total 1.430s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.048ms, total 1.430s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.428ms, total 1.430s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.076ms, total 1.430s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.049ms, total 1.430s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.090ms, total 1.431s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.162ms, total 1.431s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.063ms, total 1.431s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.321ms, total 1.431s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.072ms, total 1.431s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.035ms, total 1.431s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.092ms, total 1.432s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.159ms, total 1.432s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.041ms, total 1.432s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.362ms, total 1.432s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 3.085ms, total 1.432s, tagged coverage 79.2 2020-01-07 21:48:24 root: INFO: optimize Compile: after: IR size 21: (InsertFields (Ref global) None (__cols (ArrayMap __iruid_6459 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6460 (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6459)) (InsertFields (Ref __iruid_6460) None (__uid_41 (Cast Float64 (GetField CaffeineConsumption (GetField pheno (Ref __iruid_6460))))) (__uid_42 (F64 1.0))))))) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C72. 2020-01-07 21:48:24 root: INFO: instruction count: 44: is/hail/codegen/generated/C72.apply 2020-01-07 21:48:24 root: INFO: instruction count: 16: is/hail/codegen/generated/C72.apply 2020-01-07 21:48:24 root: INFO: instruction count: 493: is/hail/codegen/generated/C72.m1_method 2020-01-07 21:48:24 root: INFO: instruction count: 160: is/hail/codegen/generated/C72.m2_method 2020-01-07 21:48:24 root: INFO: instruction count: 14: is/hail/codegen/generated/C72.setPartitionIndex 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C72.addPartitionRegion 2020-01-07 21:48:24 root: INFO: optimize Compile: before: IR size 20: (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap __iruid_6284 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6285 (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6284)) (InsertFields (Ref __iruid_6285) None (__uid_43 (Cast Float64 (Apply nNonRefAlleles Int32 (GetField GT (Ref __iruid_6285)))))))))) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.134ms, total 1.438s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.044ms, total 1.438s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.113ms, total 1.438s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.237ms, total 1.439s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.053ms, total 1.439s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.432ms, total 1.439s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.094ms, total 1.439s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.037ms, total 1.439s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.096ms, total 1.440s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.157ms, total 1.440s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.041ms, total 1.440s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.413ms, total 1.440s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.109ms, total 1.440s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.052ms, total 1.440s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.104ms, total 1.441s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.161ms, total 1.441s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.062ms, total 1.441s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.407ms, total 1.441s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 3.443ms, total 1.441s, tagged coverage 79.8 2020-01-07 21:48:24 root: INFO: optimize Compile: after: IR size 20: (InsertFields (Ref row) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (ArrayMap __iruid_6466 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (Let __iruid_6467 (ArrayRef (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (Ref __iruid_6466)) (InsertFields (Ref __iruid_6467) None (__uid_43 (Cast Float64 (Apply nNonRefAlleles Int32 (GetField GT (Ref __iruid_6467)))))))))) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C73. 2020-01-07 21:48:24 root: INFO: instruction count: 46: is/hail/codegen/generated/C73.apply 2020-01-07 21:48:24 root: INFO: instruction count: 22: is/hail/codegen/generated/C73.apply 2020-01-07 21:48:24 root: INFO: instruction count: 589: is/hail/codegen/generated/C73.m1_method 2020-01-07 21:48:24 root: INFO: instruction count: 115: is/hail/codegen/generated/C73.m2_method 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C73.m3_method 2020-01-07 21:48:24 root: INFO: instruction count: 14: is/hail/codegen/generated/C73.setPartitionIndex 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C73.addPartitionRegion 2020-01-07 21:48:24 root: INFO: encoder cache miss (12 hits, 18 misses, 0.400) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_74. 2020-01-07 21:48:24 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_74.apply 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_74.m0_ENCODE_o_tuple_of_o_array_of_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDANDo_float64ANDo_float64ENDEND_TO_o_struct_of_o_array_of_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDANDo_float64ANDo_float64ENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_74.m1_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 74: is/hail/codegen/generated/C_etypeEncode_74.m2_ENCODE_o_array_of_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDANDo_float64ANDo_float64END_TO_o_array_of_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDANDo_float64ANDo_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_74.m3_ENCODE_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDANDo_float64ANDo_float64END_TO_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDANDo_float64ANDo_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 71: is/hail/codegen/generated/C_etypeEncode_74.m4_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_74.m5_ENCODE_o_tuple_of_o_int32END_TO_o_struct_of_o_int32END 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_74.m6_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_74.m7_ENCODE_o_int32_TO_o_int32 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_74.m8_ENCODE_o_tuple_of_o_tuple_of_o_float64ENDANDo_float64END_TO_o_struct_of_o_struct_of_o_float64ENDANDo_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 37: is/hail/codegen/generated/C_etypeEncode_74.m9_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_74.m10_ENCODE_o_tuple_of_o_float64END_TO_o_struct_of_o_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_74.m11_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_74.m12_ENCODE_o_float64_TO_o_float64 2020-01-07 21:48:24 root: INFO: decoder cache miss (41 hits, 14 misses, 0.745 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeDecode_75. 2020-01-07 21:48:24 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeDecode_75.apply 2020-01-07 21:48:24 root: INFO: instruction count: 27: is/hail/codegen/generated/C_etypeDecode_75.m0_DECODE_o_struct_of_o_array_of_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDANDo_float64ANDo_float64ENDEND_TO_o_tuple_of_o_array_of_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDANDo_float64ANDo_float64ENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_75.m1_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 97: is/hail/codegen/generated/C_etypeDecode_75.m2_INPLACE_DECODE_o_array_of_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDANDo_float64ANDo_float64END_TO_o_array_of_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDANDo_float64ANDo_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_75.m3_INPLACE_DECODE_o_struct_of_o_struct_of_o_int32ENDANDo_struct_of_o_struct_of_o_float64ENDANDo_float64ENDANDo_float64ANDo_float64END_TO_o_tuple_of_o_tuple_of_o_int32ENDANDo_tuple_of_o_tuple_of_o_float64ENDANDo_float64ENDANDo_float64ANDo_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 103: is/hail/codegen/generated/C_etypeDecode_75.m4_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_75.m5_INPLACE_DECODE_o_struct_of_o_int32END_TO_o_tuple_of_o_int32END 2020-01-07 21:48:24 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_75.m6_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_75.m7_INPLACE_DECODE_o_int32_TO_o_int32 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_75.m8_INPLACE_DECODE_o_struct_of_o_struct_of_o_float64ENDANDo_float64END_TO_o_tuple_of_o_tuple_of_o_float64ENDANDo_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 53: is/hail/codegen/generated/C_etypeDecode_75.m9_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_75.m10_INPLACE_DECODE_o_struct_of_o_float64END_TO_o_tuple_of_o_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_75.m11_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_75.m12_INPLACE_DECODE_o_float64_TO_o_float64 2020-01-07 21:48:24 MemoryStore: INFO: Block broadcast_37 stored as values in memory (estimated size 15.4 KB, free 365.5 MB) 2020-01-07 21:48:24 MemoryStore: INFO: Block broadcast_37_piece0 stored as bytes in memory (estimated size 7.8 KB, free 365.5 MB) 2020-01-07 21:48:24 BlockManagerInfo: INFO: Added broadcast_37_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 7.8 KB, free: 366.2 MB) 2020-01-07 21:48:24 SparkContext: INFO: Created broadcast 37 from broadcast at SparkBackend.scala:21 2020-01-07 21:48:24 root: INFO: optimize Compile: before: IR size 14: (InsertFields (Ref global) None (__cols (ArrayMap __iruid_6286 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (SelectFields (__uid_41 __uid_42) (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6286)))))) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.093ms, total 1.462s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.042ms, total 1.462s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.094ms, total 1.462s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.135ms, total 1.462s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.040ms, total 1.462s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.289ms, total 1.463s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.065ms, total 1.463s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.033ms, total 1.463s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.070ms, total 1.463s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.110ms, total 1.463s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.034ms, total 1.463s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.209ms, total 1.463s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.059ms, total 1.464s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.025ms, total 1.464s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.066ms, total 1.464s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.105ms, total 1.464s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.049ms, total 1.464s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.207ms, total 1.464s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 2.339ms, total 1.464s, tagged coverage 73.6 2020-01-07 21:48:24 root: INFO: optimize Compile: after: IR size 14: (InsertFields (Ref global) None (__cols (ArrayMap __iruid_6470 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (SelectFields (__uid_41 __uid_42) (ArrayRef (GetField __cols (Ref global)) (Ref __iruid_6470)))))) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C76. 2020-01-07 21:48:24 root: INFO: instruction count: 44: is/hail/codegen/generated/C76.apply 2020-01-07 21:48:24 root: INFO: instruction count: 16: is/hail/codegen/generated/C76.apply 2020-01-07 21:48:24 root: INFO: instruction count: 529: is/hail/codegen/generated/C76.m1_method 2020-01-07 21:48:24 root: INFO: instruction count: 14: is/hail/codegen/generated/C76.setPartitionIndex 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C76.addPartitionRegion 2020-01-07 21:48:24 root: INFO: optimize Compile: before: IR size 19: (Let __iruid_6287 (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (MakeStruct (locus (GetField locus (Ref row))) (alleles (GetField alleles (Ref row))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (ArrayMap __iruid_6288 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (SelectFields (__uid_43) (ArrayRef (Ref __iruid_6287) (Ref __iruid_6288))))))) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.081ms, total 1.469s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.033ms, total 1.469s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.090ms, total 1.469s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.171ms, total 1.469s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.038ms, total 1.469s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.286ms, total 1.469s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.078ms, total 1.470s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.031ms, total 1.470s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.080ms, total 1.470s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.169ms, total 1.470s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.043ms, total 1.470s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.270ms, total 1.470s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.062ms, total 1.470s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.029ms, total 1.470s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.074ms, total 1.471s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.133ms, total 1.471s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.034ms, total 1.471s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.280ms, total 1.471s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 2.592ms, total 1.471s, tagged coverage 76.5 2020-01-07 21:48:24 root: INFO: optimize Compile: after: IR size 19: (Let __iruid_6476 (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)) (MakeStruct (locus (GetField locus (Ref row))) (alleles (GetField alleles (Ref row))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (ArrayMap __iruid_6477 (ArrayRange (I32 0) (ArrayLen (GetField __cols (Ref global))) (I32 1)) (SelectFields (__uid_43) (ArrayRef (Ref __iruid_6476) (Ref __iruid_6477))))))) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C77. 2020-01-07 21:48:24 root: INFO: instruction count: 89: is/hail/codegen/generated/C77.apply 2020-01-07 21:48:24 root: INFO: instruction count: 22: is/hail/codegen/generated/C77.apply 2020-01-07 21:48:24 root: INFO: instruction count: 559: is/hail/codegen/generated/C77.m1_method 2020-01-07 21:48:24 root: INFO: instruction count: 14: is/hail/codegen/generated/C77.setPartitionIndex 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C77.addPartitionRegion 2020-01-07 21:48:24 root: INFO: encoder cache miss (12 hits, 19 misses, 0.387) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeEncode_78. 2020-01-07 21:48:24 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeEncode_78.apply 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_78.m0_ENCODE_o_tuple_of_o_array_of_o_tuple_of_o_float64ANDo_float64ENDEND_TO_o_struct_of_o_array_of_o_struct_of_o_float64ANDo_float64ENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeEncode_78.m1_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 74: is/hail/codegen/generated/C_etypeEncode_78.m2_ENCODE_o_array_of_o_tuple_of_o_float64ANDo_float64END_TO_o_array_of_o_struct_of_o_float64ANDo_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 11: is/hail/codegen/generated/C_etypeEncode_78.m3_ENCODE_o_tuple_of_o_float64ANDo_float64END_TO_o_struct_of_o_float64ANDo_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 39: is/hail/codegen/generated/C_etypeEncode_78.m4_write_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C_etypeEncode_78.m5_ENCODE_o_float64_TO_o_float64 2020-01-07 21:48:24 root: INFO: decoder cache miss (41 hits, 15 misses, 0.732 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C_etypeDecode_79. 2020-01-07 21:48:24 root: INFO: instruction count: 7: is/hail/codegen/generated/C_etypeDecode_79.apply 2020-01-07 21:48:24 root: INFO: instruction count: 27: is/hail/codegen/generated/C_etypeDecode_79.m0_DECODE_o_struct_of_o_array_of_o_struct_of_o_float64ANDo_float64ENDEND_TO_o_tuple_of_o_array_of_o_tuple_of_o_float64ANDo_float64ENDEND 2020-01-07 21:48:24 root: INFO: instruction count: 28: is/hail/codegen/generated/C_etypeDecode_79.m1_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 97: is/hail/codegen/generated/C_etypeDecode_79.m2_INPLACE_DECODE_o_array_of_o_struct_of_o_float64ANDo_float64END_TO_o_array_of_o_tuple_of_o_float64ANDo_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 21: is/hail/codegen/generated/C_etypeDecode_79.m3_INPLACE_DECODE_o_struct_of_o_float64ANDo_float64END_TO_o_tuple_of_o_float64ANDo_float64END 2020-01-07 21:48:24 root: INFO: instruction count: 53: is/hail/codegen/generated/C_etypeDecode_79.m4_read_fields_group_0 2020-01-07 21:48:24 root: INFO: instruction count: 8: is/hail/codegen/generated/C_etypeDecode_79.m5_INPLACE_DECODE_o_float64_TO_o_float64 2020-01-07 21:48:24 MemoryStore: INFO: Block broadcast_38 stored as values in memory (estimated size 6.1 KB, free 365.5 MB) 2020-01-07 21:48:24 MemoryStore: INFO: Block broadcast_38_piece0 stored as bytes in memory (estimated size 2.9 KB, free 365.5 MB) 2020-01-07 21:48:24 BlockManagerInfo: INFO: Added broadcast_38_piece0 in memory on sinin.kps.ku.ac.th:35729 (size: 2.9 KB, free: 366.2 MB) 2020-01-07 21:48:24 SparkContext: INFO: Created broadcast 38 from broadcast at SparkBackend.scala:21 2020-01-07 21:48:24 root: INFO: optimize Compile: before: IR size 2: (CastRename Struct{__cols:Array[Struct{__y_0:Float64,__cov0:Float64}]} (Ref global)) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.034ms, total 1.490s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.011ms, total 1.491s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.027ms, total 1.491s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.087ms, total 1.491s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.016ms, total 1.491s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.079ms, total 1.491s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 0.499ms, total 1.491s, tagged coverage 51.0 2020-01-07 21:48:24 root: INFO: optimize Compile: after: IR size 2: (CastRename Struct{__cols:Array[Struct{__y_0:Float64,__cov0:Float64}]} (Ref global)) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C80. 2020-01-07 21:48:24 root: INFO: instruction count: 16: is/hail/codegen/generated/C80.apply 2020-01-07 21:48:24 root: INFO: instruction count: 16: is/hail/codegen/generated/C80.apply 2020-01-07 21:48:24 root: INFO: instruction count: 14: is/hail/codegen/generated/C80.setPartitionIndex 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C80.addPartitionRegion 2020-01-07 21:48:24 root: INFO: optimize Compile: before: IR size 2: (CastRename Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{__uid_40:Float64}]} (Ref row)) 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- FoldConstants : 0.025ms, total 1.493s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ExtractIntervalFilters : 0.007ms, total 1.493s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- Simplify : 0.017ms, total 1.493s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardLets : 0.053ms, total 1.493s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- ForwardRelationalLets : 0.012ms, total 1.493s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize -- PruneDeadFields : 0.067ms, total 1.493s 2020-01-07 21:48:24 root: INFO: Timer: Time taken for Optimize : 0.356ms, total 1.493s, tagged coverage 50.9 2020-01-07 21:48:24 root: INFO: optimize Compile: after: IR size 2: (CastRename Struct{locus:Locus(GRCh37),alleles:Array[String],`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{__uid_40:Float64}]} (Ref row)) 2020-01-07 21:48:24 root: INFO: instruction count: 3: is/hail/codegen/generated/C81. 2020-01-07 21:48:24 root: INFO: instruction count: 16: is/hail/codegen/generated/C81.apply 2020-01-07 21:48:24 root: INFO: instruction count: 22: is/hail/codegen/generated/C81.apply 2020-01-07 21:48:24 root: INFO: instruction count: 14: is/hail/codegen/generated/C81.setPartitionIndex 2020-01-07 21:48:24 root: INFO: instruction count: 6: is/hail/codegen/generated/C81.addPartitionRegion 2020-01-07 21:48:24 Hail: INFO: linear_regression_rows: running on 250 samples for 1 response variable y, with input variable x, and 1 additional covariate... 2020-01-07 21:48:24 JniLoader: INFO: successfully loaded /tmp/jniloader4017248586949720161netlib-native_system-linux-x86_64.so 2020-01-07 21:48:24 JniLoader: INFO: already loaded netlib-native_system-linux-x86_64.so