2021-06-28 06:59:06 Hail: INFO: Running Hail version 0.2.67-a673309b0445 2021-06-28 06:59:07 SparkContext: WARN: Using an existing SparkContext; some configuration may not take effect. 2021-06-28 06:59:08 root: INFO: RegionPool: initialized for thread 15: Thread-5 2021-06-28 06:59:08 root: INFO: globbing path /n/holylfs/LABS/Projects/UKBiobank/ukb_imp_chr[1-22]_v3.bgen returned 2 files: ukb_imp_chr1_v3.bgen,ukb_imp_chr2_v3.bgen 2021-06-28 06:59:09 MemoryStore: INFO: Block broadcast_0 stored as values in memory (estimated size 50.0 KiB, free 57.8 GiB) 2021-06-28 06:59:10 MemoryStore: INFO: Block broadcast_0_piece0 stored as bytes in memory (estimated size 3.2 KiB, free 57.8 GiB) 2021-06-28 06:59:10 BlockManagerInfo: INFO: Added broadcast_0_piece0 in memory on holy7c02601.rc.fas.harvard.edu:33255 (size: 3.2 KiB, free: 57.8 GiB) 2021-06-28 06:59:10 SparkContext: INFO: Created broadcast 0 from broadcast at SparkBackend.scala:275 2021-06-28 06:59:10 root: INFO: decoder cache miss (0 hits, 1 misses, 0.000 2021-06-28 06:59:10 root: INFO: instruction count: 3: __C1FSContainer. 2021-06-28 06:59:10 root: INFO: instruction count: 3: __C1FSContainer. 2021-06-28 06:59:10 root: INFO: instruction count: 3: __C2etypeDecode. 2021-06-28 06:59:10 root: INFO: instruction count: 5: __C2etypeDecode.apply 2021-06-28 06:59:10 root: INFO: instruction count: 30: __C2etypeDecode.__m3DECODE_o_struct_of_r_int64ANDr_array_of_r_struct_of_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_ENDENDEND_TO_o_tuple_of_r_int64ANDr_array_of_r_tuple_of_o_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_ENDENDEND 2021-06-28 06:59:10 root: INFO: instruction count: 10: __C2etypeDecode.__m4INPLACE_DECODE_r_int64_TO_r_int64 2021-06-28 06:59:10 root: INFO: instruction count: 64: __C2etypeDecode.__m5INPLACE_DECODE_r_array_of_r_struct_of_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_ENDEND_TO_r_array_of_r_tuple_of_o_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_ENDEND 2021-06-28 06:59:10 root: INFO: instruction count: 60: __C2etypeDecode.__m6INPLACE_DECODE_r_struct_of_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_ENDEND_TO_r_tuple_of_o_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_ENDEND 2021-06-28 06:59:10 root: INFO: instruction count: 71: __C2etypeDecode.__m7INPLACE_DECODE_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryEND_TO_o_tuple_of_o_locusANDo_array_of_o_stringEND 2021-06-28 06:59:10 root: INFO: instruction count: 22: __C2etypeDecode.__m8INPLACE_DECODE_o_struct_of_r_binaryANDr_int32END_TO_o_locus 2021-06-28 06:59:10 root: INFO: instruction count: 33: __C2etypeDecode.__m9INPLACE_DECODE_r_binary_TO_r_string 2021-06-28 06:59:10 root: INFO: instruction count: 10: __C2etypeDecode.__m10INPLACE_DECODE_r_int32_TO_r_int32 2021-06-28 06:59:10 root: INFO: instruction count: 120: __C2etypeDecode.__m11INPLACE_DECODE_o_array_of_o_binary_TO_o_array_of_o_string 2021-06-28 06:59:10 root: INFO: instruction count: 33: __C2etypeDecode.__m12INPLACE_DECODE_o_binary_TO_o_string 2021-06-28 06:59:10 root: INFO: instruction count: 8: __C2etypeDecode.__m13INPLACE_DECODE_r_struct_of_END_TO_r_tuple_of_END 2021-06-28 06:59:10 root: INFO: decoder cache miss (0 hits, 2 misses, 0.000 2021-06-28 06:59:10 root: INFO: instruction count: 3: __C14FSContainer. 2021-06-28 06:59:10 root: INFO: instruction count: 3: __C14FSContainer. 2021-06-28 06:59:10 root: INFO: instruction count: 3: __C15etypeDecode. 2021-06-28 06:59:10 root: INFO: instruction count: 5: __C15etypeDecode.apply 2021-06-28 06:59:10 root: INFO: instruction count: 23: __C15etypeDecode.__m16DECODE_o_struct_of_r_array_of_r_struct_of_r_int64ANDr_int64ANDo_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_ENDENDEND_TO_o_tuple_of_r_array_of_r_tuple_of_r_int64ANDr_int64ANDo_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_ENDENDEND 2021-06-28 06:59:10 root: INFO: instruction count: 64: __C15etypeDecode.__m17INPLACE_DECODE_r_array_of_r_struct_of_r_int64ANDr_int64ANDo_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_ENDEND_TO_r_array_of_r_tuple_of_r_int64ANDr_int64ANDo_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_ENDEND 2021-06-28 06:59:10 root: INFO: instruction count: 74: __C15etypeDecode.__m18INPLACE_DECODE_r_struct_of_r_int64ANDr_int64ANDo_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_ENDEND_TO_r_tuple_of_r_int64ANDr_int64ANDo_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_ENDEND 2021-06-28 06:59:10 root: INFO: instruction count: 10: __C15etypeDecode.__m19INPLACE_DECODE_r_int64_TO_r_int64 2021-06-28 06:59:10 root: INFO: instruction count: 71: __C15etypeDecode.__m20INPLACE_DECODE_o_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryEND_TO_o_tuple_of_o_locusANDo_array_of_o_stringEND 2021-06-28 06:59:10 root: INFO: instruction count: 22: __C15etypeDecode.__m21INPLACE_DECODE_o_struct_of_r_binaryANDr_int32END_TO_o_locus 2021-06-28 06:59:10 root: INFO: instruction count: 33: __C15etypeDecode.__m22INPLACE_DECODE_r_binary_TO_r_string 2021-06-28 06:59:10 root: INFO: instruction count: 10: __C15etypeDecode.__m23INPLACE_DECODE_r_int32_TO_r_int32 2021-06-28 06:59:10 root: INFO: instruction count: 120: __C15etypeDecode.__m24INPLACE_DECODE_o_array_of_o_binary_TO_o_array_of_o_string 2021-06-28 06:59:10 root: INFO: instruction count: 33: __C15etypeDecode.__m25INPLACE_DECODE_o_binary_TO_o_string 2021-06-28 06:59:10 root: INFO: instruction count: 8: __C15etypeDecode.__m26INPLACE_DECODE_r_struct_of_END_TO_r_tuple_of_END 2021-06-28 06:59:10 root: INFO: RegionPool: REPORT_THRESHOLD: 305.0K allocated (192.0K blocks / 113.0K chunks), regions.size = 2, 0 current java objects, thread 15: Thread-5 2021-06-28 06:59:10 root: INFO: RegionPool: REPORT_THRESHOLD: 512.0K allocated (320.0K blocks / 192.0K chunks), regions.size = 2, 0 current java objects, thread 15: Thread-5 2021-06-28 06:59:10 root: INFO: Index reader cache queries: 4 2021-06-28 06:59:10 root: INFO: Index reader cache hit rate: 0.25 2021-06-28 06:59:10 root: INFO: decoder cache hit 2021-06-28 06:59:10 root: INFO: decoder cache hit 2021-06-28 06:59:11 root: INFO: Index reader cache queries: 4 2021-06-28 06:59:11 root: INFO: Index reader cache hit rate: 0.25 2021-06-28 06:59:11 Hail: WARN: BGEN file 'file:/n/holylfs/LABS/liang_lab/Projects/UKBiobank/ukb_imp_chr1_v3.bgen' contains no sample ID block and no sample ID file given. Using _0, _1, ..., _N as sample IDs. 2021-06-28 06:59:12 Hail: INFO: Number of BGEN files parsed: 2 2021-06-28 06:59:12 Hail: INFO: Number of samples in BGEN files: 487409 2021-06-28 06:59:12 Hail: INFO: Number of variants across all BGEN files: 15531854 2021-06-28 06:59:12 MemoryStore: INFO: Block broadcast_1 stored as values in memory (estimated size 370.8 KiB, free 57.8 GiB) 2021-06-28 06:59:12 MemoryStore: INFO: Block broadcast_1_piece0 stored as bytes in memory (estimated size 27.1 KiB, free 57.8 GiB) 2021-06-28 06:59:12 BlockManagerInfo: INFO: Added broadcast_1_piece0 in memory on holy7c02601.rc.fas.harvard.edu:33255 (size: 27.1 KiB, free: 57.8 GiB) 2021-06-28 06:59:12 SparkContext: INFO: Created broadcast 1 from broadcast at SparkBackend.scala:275 2021-06-28 06:59:12 root: INFO: decoder cache hit 2021-06-28 06:59:12 root: INFO: decoder cache hit 2021-06-28 06:59:13 root: INFO: Index reader cache queries: 944 2021-06-28 06:59:13 root: INFO: Index reader cache hit rate: 0.7478813559322034 2021-06-28 06:59:18 root: INFO: Index reader cache queries: 980 2021-06-28 06:59:18 root: INFO: Index reader cache hit rate: 0.7479591836734694 2021-06-28 06:59:18 root: INFO: RegionPool: FREE: 512.0K allocated (512.0K blocks / 0 chunks), regions.size = 5, 0 current java objects, thread 15: Thread-5 2021-06-28 06:59:18 root: INFO: timing SparkBackend.parse_matrix_ir total 10.276s self 10.276s children 0.000ms %children 0.00% 2021-06-28 06:59:18 root: INFO: RegionPool: initialized for thread 15: Thread-5 2021-06-28 06:59:18 root: INFO: RegionPool: FREE: 64.0K allocated (64.0K blocks / 0 chunks), regions.size = 1, 0 current java objects, thread 15: Thread-5 2021-06-28 06:59:18 root: INFO: timing SparkBackend.parse_value_ir total 141.226ms self 141.226ms children 0.000ms %children 0.00% 2021-06-28 06:59:18 root: INFO: RegionPool: initialized for thread 15: Thread-5 2021-06-28 06:59:18 root: INFO: starting execution of query hail_query_1 of initial size 21 2021-06-28 06:59:19 root: INFO: optimize optimize: relationalLowerer, initial IR: before: IR size 21: (MatrixWrite "{\"name\":\"MatrixNativeWriter\",\"path\":\"ukb_merged_bgen_27Jun2021.mt\",\"overwrite\":true,\"stageLocally\":false,\"codecSpecJSONStr\":null,\"partitions\":null,\"partitionsTypeStr\":null,\"checkpointFile\":null}" (MatrixMapEntries (MatrixMapRows (MatrixRead None False False "{\"name\":\"MatrixBGENReader\",\"files\":[\"/n/holylfs/LABS/liang_lab/Projects/UKBiobank/ukb_imp_chr[1-22]_v3.bgen\"],\"sampleFile\":null,\"indexFileMap\":{},\"nPartitions\":480,\"blockSizeInMB\":null,\"includedVariants\":null}") (SelectFields (locus alleles rsid varid) (MakeStruct (locus (GetField locus (Ref va))) (alleles (GetField alleles (Ref va))) (rsid (GetField rsid (Ref va))) (varid (GetField varid (Ref va))) (offset (GetField offset (Ref va))) (file_idx (GetField file_idx (Ref va)))))) (SelectFields (GP) (SelectFields (GT GP dosage) (Ref g))))) 2021-06-28 06:59:19 root: INFO: optimize optimize: relationalLowerer, initial IR: after: IR size 12: (MatrixWrite "{\"name\":\"MatrixNativeWriter\",\"path\":\"ukb_merged_bgen_27Jun2021.mt\",\"overwrite\":true,\"stageLocally\":false,\"codecSpecJSONStr\":null,\"partitions\":null,\"partitionsTypeStr\":null,\"checkpointFile\":null}" (MatrixMapRows (MatrixRead Matrix{global:Struct{},col_key:[s],col:Struct{s:String},row_key:[[locus,alleles]],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,varid:String},entry:Struct{GP:Array[Float64]}} False False "{\"name\":\"MatrixBGENReader\",\"files\":[\"/n/holylfs/LABS/liang_lab/Projects/UKBiobank/ukb_imp_chr[1-22]_v3.bgen\"],\"sampleFile\":null,\"indexFileMap\":{},\"nPartitions\":480,\"blockSizeInMB\":null,\"includedVariants\":null}") (MakeStruct (locus (GetField locus (Ref va))) (alleles (GetField alleles (Ref va))) (rsid (GetField rsid (Ref va))) (varid (GetField varid (Ref va)))))) 2021-06-28 06:59:19 root: INFO: optimize optimize: relationalLowerer, after LowerMatrixToTable: before: IR size 25: (TableWrite "{\"writer\":{\"path\":\"ukb_merged_bgen_27Jun2021.mt\",\"overwrite\":true,\"stageLocally\":false,\"codecSpecJSONStr\":null,\"partitions\":null,\"partitionsTypeStr\":null,\"checkpointFile\":null},\"colsFieldName\":\"__cols\",\"entriesFieldName\":\"the entries! [877f12a8827e18f61222c6c8c5fb04a8]\",\"colKey\":[\"s\"]}" (TableMapRows (TableRead Table{global:Struct{__cols:Array[Struct{s:String}]},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,varid:String,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GP:Array[Float64]}]}} False "{\"name\":\"MatrixBGENReader\",\"files\":[\"/n/holylfs/LABS/liang_lab/Projects/UKBiobank/ukb_imp_chr[1-22]_v3.bgen\"],\"sampleFile\":null,\"indexFileMap\":{},\"nPartitions\":480,\"blockSizeInMB\":null,\"includedVariants\":null}") (Let n_cols (ArrayLen (GetField __cols (Ref global))) (InsertFields (Let __iruid_0 (MakeStruct) (MakeStruct (locus (GetField locus (SelectFields (locus alleles rsid varid) (Ref row)))) (alleles (GetField alleles (SelectFields (locus alleles rsid varid) (Ref row)))) (rsid (GetField rsid (SelectFields (locus alleles rsid varid) (Ref row)))) (varid (GetField varid (SelectFields (locus alleles rsid varid) (Ref row)))))) None (`the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row))))))) 2021-06-28 06:59:19 root: INFO: optimize optimize: relationalLowerer, after LowerMatrixToTable: after: IR size 14: (TableWrite "{\"writer\":{\"path\":\"ukb_merged_bgen_27Jun2021.mt\",\"overwrite\":true,\"stageLocally\":false,\"codecSpecJSONStr\":null,\"partitions\":null,\"partitionsTypeStr\":null,\"checkpointFile\":null},\"colsFieldName\":\"__cols\",\"entriesFieldName\":\"the entries! [877f12a8827e18f61222c6c8c5fb04a8]\",\"colKey\":[\"s\"]}" (TableMapRows (TableRead Table{global:Struct{__cols:Array[Struct{s:String}]},key:[locus,alleles],row:Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,varid:String,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GP:Array[Float64]}]}} False "{\"name\":\"MatrixBGENReader\",\"files\":[\"/n/holylfs/LABS/liang_lab/Projects/UKBiobank/ukb_imp_chr[1-22]_v3.bgen\"],\"sampleFile\":null,\"indexFileMap\":{},\"nPartitions\":480,\"blockSizeInMB\":null,\"includedVariants\":null}") (MakeStruct (locus (GetField locus (Ref row))) (alleles (GetField alleles (Ref row))) (rsid (GetField rsid (Ref row))) (varid (GetField varid (Ref row))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (Ref row)))))) 2021-06-28 06:59:19 root: INFO: interpreting non compilable node: TableWrite 2021-06-28 06:59:19 root: INFO: instruction count: 3: __C27FSContainer. 2021-06-28 06:59:19 root: INFO: instruction count: 3: __C27FSContainer. 2021-06-28 06:59:19 root: INFO: instruction count: 3: __C28bgen_rdd_decoder. 2021-06-28 06:59:19 root: INFO: instruction count: 611: __C28bgen_rdd_decoder.apply 2021-06-28 06:59:19 root: INFO: instruction count: 16: __C28bgen_rdd_decoder.apply 2021-06-28 06:59:19 root: INFO: instruction count: 221: __C28bgen_rdd_decoder.__m42memoizeEntries 2021-06-28 06:59:19 root: INFO: instruction count: 9: __C28bgen_rdd_decoder.setPartitionIndex 2021-06-28 06:59:19 root: INFO: instruction count: 4: __C28bgen_rdd_decoder.addPartitionRegion 2021-06-28 06:59:19 root: INFO: instruction count: 4: __C28bgen_rdd_decoder.setPool 2021-06-28 06:59:19 root: INFO: instruction count: 3: __C28bgen_rdd_decoder.addFS 2021-06-28 06:59:19 root: INFO: instruction count: 568: __C28bgen_rdd_decoder.applyregion25_58 2021-06-28 06:59:19 root: INFO: instruction count: 3: __C46applySpills. 2021-06-28 06:59:19 root: INFO: decoder cache hit 2021-06-28 06:59:19 root: INFO: decoder cache hit 2021-06-28 06:59:20 root: INFO: RegionPool: REPORT_THRESHOLD: 3.8M allocated (128.0K blocks / 3.7M chunks), regions.size = 2, 0 current java objects, thread 15: Thread-5 2021-06-28 06:59:20 root: INFO: RegionPool: REPORT_THRESHOLD: 3.9M allocated (192.0K blocks / 3.7M chunks), regions.size = 2, 0 current java objects, thread 15: Thread-5 2021-06-28 06:59:20 root: INFO: RegionPool: REPORT_THRESHOLD: 4.0M allocated (256.0K blocks / 3.7M chunks), regions.size = 2, 0 current java objects, thread 15: Thread-5 2021-06-28 06:59:20 root: INFO: RegionPool: REPORT_THRESHOLD: 4.0M allocated (320.0K blocks / 3.7M chunks), regions.size = 2, 0 current java objects, thread 15: Thread-5 2021-06-28 06:59:20 root: INFO: RegionPool: REPORT_THRESHOLD: 4.1M allocated (384.0K blocks / 3.7M chunks), regions.size = 2, 0 current java objects, thread 15: Thread-5 2021-06-28 06:59:20 root: INFO: RegionPool: REPORT_THRESHOLD: 8.0M allocated (4.3M blocks / 3.7M chunks), regions.size = 2, 0 current java objects, thread 15: Thread-5 2021-06-28 06:59:21 root: INFO: optimize optimize: compileLowerer, initial IR: before: IR size 14: (Coalesce (MakeStruct (locus (GetField locus (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (alleles (GetField alleles (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (rsid (GetField rsid (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (varid (GetField varid (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1)))) (Die Struct{locus:Locus(GRCh37),alleles:Array[String],rsid:String,varid:String,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:Array[Struct{GP:Array[Float64]}]} -1 (Str "Internal e..."))) 2021-06-28 06:59:21 root: INFO: optimize optimize: compileLowerer, initial IR: after: IR size 11: (MakeStruct (locus (GetField locus (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (alleles (GetField alleles (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (rsid (GetField rsid (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (varid (GetField varid (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1)))) 2021-06-28 06:59:21 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: before: IR size 11: (MakeStruct (locus (GetField locus (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (alleles (GetField alleles (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (rsid (GetField rsid (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (varid (GetField varid (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1)))) 2021-06-28 06:59:21 root: INFO: optimize optimize: compileLowerer, after InlineApplyIR: after: IR size 11: (MakeStruct (locus (GetField locus (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (alleles (GetField alleles (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (rsid (GetField rsid (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (varid (GetField varid (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1)))) 2021-06-28 06:59:21 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: before: IR size 11: (MakeStruct (locus (GetField locus (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (alleles (GetField alleles (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (rsid (GetField rsid (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (varid (GetField varid (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1)))) 2021-06-28 06:59:21 root: INFO: optimize optimize: compileLowerer, after LowerArrayAggsToRunAggs: after: IR size 11: (MakeStruct (locus (GetField locus (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (alleles (GetField alleles (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (rsid (GetField rsid (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (varid (GetField varid (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1))) (the entries! [877f12a8827e18f61222c6c8c5fb04a8] (GetField `the entries! [877f12a8827e18f61222c6c8c5fb04a8]` (In SingleCodeEmitParamType(true, PTypeReferenceSingleCodeType(+PCStruct{locus:PCLocus(GRCh37),alleles:PCArray[PCString],rsid:PCString,varid:PCString,`the entries! [877f12a8827e18f61222c6c8c5fb04a8]`:PCArray[PCStruct{GP:+PCArray[+PFloat64]}]})) 1)))) 2021-06-28 06:59:21 root: INFO: instruction count: 3: __C61FSContainer. 2021-06-28 06:59:21 root: INFO: instruction count: 3: __C61FSContainer. 2021-06-28 06:59:21 root: INFO: instruction count: 3: __C62Compiled. 2021-06-28 06:59:21 root: INFO: instruction count: 155: __C62Compiled.apply 2021-06-28 06:59:21 root: INFO: instruction count: 9: __C62Compiled.setPartitionIndex 2021-06-28 06:59:21 root: INFO: instruction count: 4: __C62Compiled.addPartitionRegion 2021-06-28 06:59:21 root: INFO: instruction count: 4: __C62Compiled.setPool 2021-06-28 06:59:21 root: INFO: instruction count: 3: __C62Compiled.addFS 2021-06-28 06:59:21 root: INFO: encoder cache miss (0 hits, 1 misses, 0.000) 2021-06-28 06:59:21 root: INFO: instruction count: 3: __C66FSContainer. 2021-06-28 06:59:21 root: INFO: instruction count: 3: __C66FSContainer. 2021-06-28 06:59:21 root: INFO: instruction count: 3: __C67etypeEncode. 2021-06-28 06:59:21 root: INFO: instruction count: 5: __C67etypeEncode.apply 2021-06-28 06:59:21 root: INFO: instruction count: 115: __C67etypeEncode.__m68ENCODE_r_tuple_of_o_locusANDo_array_of_o_stringANDo_stringANDo_stringANDo_array_of_o_tuple_of_r_array_of_r_float64ENDEND_TO_r_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryANDo_binaryANDo_binaryEND 2021-06-28 06:59:21 root: INFO: instruction count: 39: __C67etypeEncode.__m69ENCODE_o_locus_TO_o_struct_of_r_binaryANDr_int32END 2021-06-28 06:59:21 root: INFO: instruction count: 18: __C67etypeEncode.__m70ENCODE_r_string_TO_r_binary 2021-06-28 06:59:21 root: INFO: instruction count: 6: __C67etypeEncode.__m71ENCODE_r_int32_TO_r_int32 2021-06-28 06:59:21 root: INFO: instruction count: 125: __C67etypeEncode.__m72ENCODE_o_array_of_o_string_TO_o_array_of_o_binary 2021-06-28 06:59:21 root: INFO: instruction count: 18: __C67etypeEncode.__m73ENCODE_o_string_TO_o_binary 2021-06-28 06:59:21 root: INFO: encoder cache miss (0 hits, 2 misses, 0.000) 2021-06-28 06:59:21 root: INFO: instruction count: 3: __C74FSContainer. 2021-06-28 06:59:21 root: INFO: instruction count: 3: __C74FSContainer. 2021-06-28 06:59:21 root: INFO: instruction count: 3: __C75etypeEncode. 2021-06-28 06:59:21 root: INFO: instruction count: 5: __C75etypeEncode.apply 2021-06-28 06:59:21 root: INFO: instruction count: 35: __C75etypeEncode.__m76ENCODE_r_tuple_of_o_locusANDo_array_of_o_stringANDo_stringANDo_stringANDo_array_of_o_tuple_of_r_array_of_r_float64ENDEND_TO_r_struct_of_o_array_of_o_struct_of_r_array_of_r_float64ENDEND 2021-06-28 06:59:21 root: INFO: instruction count: 121: __C75etypeEncode.__m77ENCODE_o_array_of_o_tuple_of_r_array_of_r_float64END_TO_o_array_of_o_struct_of_r_array_of_r_float64END 2021-06-28 06:59:21 root: INFO: instruction count: 13: __C75etypeEncode.__m78ENCODE_o_tuple_of_r_array_of_r_float64END_TO_o_struct_of_r_array_of_r_float64END 2021-06-28 06:59:21 root: INFO: instruction count: 49: __C75etypeEncode.__m79ENCODE_r_array_of_r_float64_TO_r_array_of_r_float64 2021-06-28 06:59:21 root: INFO: instruction count: 6: __C75etypeEncode.__m80ENCODE_r_float64_TO_r_float64 2021-06-28 06:59:21 root: INFO: instruction count: 3: __C81FSContainer. 2021-06-28 06:59:21 root: INFO: instruction count: 3: __C81FSContainer. 2021-06-28 06:59:21 root: INFO: instruction count: 3: __C82indexwriter. 2021-06-28 06:59:21 root: INFO: instruction count: 319: __C82indexwriter.apply 2021-06-28 06:59:21 root: INFO: instruction count: 547: __C82indexwriter.__m91writeInternalNode 2021-06-28 06:59:21 root: INFO: instruction count: 13: __C82indexwriter.__m92ENCODE_o_tuple_of_r_array_of_r_tuple_of_r_int64ANDr_int64ANDr_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_o_int64ENDENDEND_TO_o_struct_of_r_array_of_r_struct_of_r_int64ANDr_int64ANDr_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDENDEND 2021-06-28 06:59:21 root: INFO: instruction count: 45: __C82indexwriter.__m93ENCODE_r_array_of_r_tuple_of_r_int64ANDr_int64ANDr_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_o_int64ENDEND_TO_r_array_of_r_struct_of_r_int64ANDr_int64ANDr_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDEND 2021-06-28 06:59:21 root: INFO: instruction count: 45: __C82indexwriter.__m94ENCODE_r_tuple_of_r_int64ANDr_int64ANDr_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_o_int64ENDEND_TO_r_struct_of_r_int64ANDr_int64ANDr_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDEND 2021-06-28 06:59:21 root: INFO: instruction count: 6: __C82indexwriter.__m95ENCODE_r_int64_TO_r_int64 2021-06-28 06:59:21 root: INFO: instruction count: 49: __C82indexwriter.__m96ENCODE_r_tuple_of_o_locusANDo_array_of_o_stringEND_TO_r_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryEND 2021-06-28 06:59:21 root: INFO: instruction count: 39: __C82indexwriter.__m97ENCODE_o_locus_TO_o_struct_of_r_binaryANDr_int32END 2021-06-28 06:59:21 root: INFO: instruction count: 18: __C82indexwriter.__m98ENCODE_r_string_TO_r_binary 2021-06-28 06:59:21 root: INFO: instruction count: 6: __C82indexwriter.__m99ENCODE_r_int32_TO_r_int32 2021-06-28 06:59:21 root: INFO: instruction count: 125: __C82indexwriter.__m100ENCODE_o_array_of_o_string_TO_o_array_of_o_binary 2021-06-28 06:59:21 root: INFO: instruction count: 18: __C82indexwriter.__m101ENCODE_o_string_TO_o_binary 2021-06-28 06:59:21 root: INFO: instruction count: 34: __C82indexwriter.__m102ENCODE_r_tuple_of_o_int64END_TO_r_struct_of_o_int64END 2021-06-28 06:59:21 root: INFO: instruction count: 6: __C82indexwriter.__m103ENCODE_o_int64_TO_o_int64 2021-06-28 06:59:21 root: INFO: instruction count: 464: __C82indexwriter.__m105writeLeafNode 2021-06-28 06:59:21 root: INFO: instruction count: 23: __C82indexwriter.__m106ENCODE_o_tuple_of_r_int64ANDr_array_of_r_tuple_of_r_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_o_int64ENDENDEND_TO_o_struct_of_r_int64ANDr_array_of_r_struct_of_r_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDENDEND 2021-06-28 06:59:21 root: INFO: instruction count: 45: __C82indexwriter.__m107ENCODE_r_array_of_r_tuple_of_r_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_o_int64ENDEND_TO_r_array_of_r_struct_of_r_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDEND 2021-06-28 06:59:21 root: INFO: instruction count: 25: __C82indexwriter.__m108ENCODE_r_tuple_of_r_tuple_of_o_locusANDo_array_of_o_stringENDANDr_int64ANDr_tuple_of_o_int64ENDEND_TO_r_struct_of_r_struct_of_o_struct_of_r_binaryANDr_int32ENDANDo_array_of_o_binaryENDANDr_int64ANDr_struct_of_o_int64ENDEND 2021-06-28 06:59:21 root: INFO: instruction count: 52: __C82indexwriter.__m109flush 2021-06-28 06:59:21 root: INFO: instruction count: 152: __C82indexwriter.init 2021-06-28 06:59:21 root: INFO: instruction count: 4: __C82indexwriter.setObjects 2021-06-28 06:59:21 root: INFO: instruction count: 28: __C82indexwriter.close 2021-06-28 06:59:21 root: INFO: instruction count: 7: __C82indexwriter.trackedOS 2021-06-28 06:59:21 root: INFO: instruction count: 9: __C82indexwriter.setPartitionIndex 2021-06-28 06:59:21 root: INFO: instruction count: 4: __C82indexwriter.addPartitionRegion 2021-06-28 06:59:21 root: INFO: instruction count: 4: __C82indexwriter.setPool 2021-06-28 06:59:21 root: INFO: instruction count: 3: __C82indexwriter.addFS 2021-06-28 06:59:21 root: INFO: instruction count: 3: __C113__m91writeInternalNodeSpills. 2021-06-28 06:59:21 SparkContext: INFO: Starting job: collect at ContextRDD.scala:176 2021-06-28 06:59:21 DAGScheduler: INFO: Got job 0 (collect at ContextRDD.scala:176) with 481 output partitions 2021-06-28 06:59:21 DAGScheduler: INFO: Final stage: ResultStage 0 (collect at ContextRDD.scala:176) 2021-06-28 06:59:21 DAGScheduler: INFO: Parents of final stage: List() 2021-06-28 06:59:21 DAGScheduler: INFO: Missing parents: List() 2021-06-28 06:59:21 DAGScheduler: INFO: Submitting ResultStage 0 (MapPartitionsRDD[5] at mapPartitions at ContextRDD.scala:168), which has no missing parents 2021-06-28 06:59:21 MemoryStore: INFO: Block broadcast_2 stored as values in memory (estimated size 106.9 KiB, free 57.8 GiB) 2021-06-28 06:59:21 MemoryStore: INFO: Block broadcast_2_piece0 stored as bytes in memory (estimated size 51.3 KiB, free 57.8 GiB) 2021-06-28 06:59:21 BlockManagerInfo: INFO: Added broadcast_2_piece0 in memory on holy7c02601.rc.fas.harvard.edu:33255 (size: 51.3 KiB, free: 57.8 GiB) 2021-06-28 06:59:21 SparkContext: INFO: Created broadcast 2 from broadcast at DAGScheduler.scala:1383 2021-06-28 06:59:21 DAGScheduler: INFO: Submitting 481 missing tasks from ResultStage 0 (MapPartitionsRDD[5] at mapPartitions at ContextRDD.scala:168) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)) 2021-06-28 06:59:21 TaskSchedulerImpl: INFO: Adding task set 0.0 with 481 tasks resource profile 0 2021-06-28 06:59:21 TaskSetManager: INFO: Starting task 0.0 in stage 0.0 (TID 0) (holy7c02601.rc.fas.harvard.edu, executor driver, partition 0, PROCESS_LOCAL, 4957 bytes) taskResourceAssignments Map() 2021-06-28 06:59:21 Executor: INFO: Running task 0.0 in stage 0.0 (TID 0) 2021-06-28 06:59:22 root: INFO: RegionPool: initialized for thread 63: Executor task launch worker for task 0.0 in stage 0.0 (TID 0) 2021-06-28 06:59:22 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), regions.size = 3, 0 current java objects, thread 63: Executor task launch worker for task 0.0 in stage 0.0 (TID 0) 2021-06-28 06:59:22 root: INFO: RegionPool: REPORT_THRESHOLD: 544.0K allocated (320.0K blocks / 224.0K chunks), regions.size = 4, 0 current java objects, thread 63: Executor task launch worker for task 0.0 in stage 0.0 (TID 0) 2021-06-28 06:59:22 root: INFO: RegionPool: REPORT_THRESHOLD: 1.0M allocated (448.0K blocks / 625.0K chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 0.0 in stage 0.0 (TID 0) 2021-06-28 06:59:22 root: INFO: RegionPool: REPORT_THRESHOLD: 2.0M allocated (1.0M blocks / 1.0M chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 0.0 in stage 0.0 (TID 0) 2021-06-28 06:59:22 root: INFO: RegionPool: REPORT_THRESHOLD: 7.1M allocated (2.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 0.0 in stage 0.0 (TID 0) 2021-06-28 07:38:38 root: INFO: RegionPool: REPORT_THRESHOLD: 8.0M allocated (3.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 0.0 in stage 0.0 (TID 0) 2021-06-28 09:33:12 root: INFO: Index reader cache queries: 16 2021-06-28 09:33:12 root: INFO: Index reader cache hit rate: 0.4375 2021-06-28 09:33:12 root: INFO: TaskReport: stage=0, partition=0, attempt=0, peakBytes=10579368, peakBytesReadable=10.09 MiB 2021-06-28 09:33:12 root: INFO: RegionPool: FREE: 5.3M allocated (5.3M blocks / 0 chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 0.0 in stage 0.0 (TID 0) 2021-06-28 09:33:12 Executor: INFO: Finished task 0.0 in stage 0.0 (TID 0). 1048 bytes result sent to driver 2021-06-28 09:33:12 TaskSetManager: INFO: Starting task 1.0 in stage 0.0 (TID 1) (holy7c02601.rc.fas.harvard.edu, executor driver, partition 1, PROCESS_LOCAL, 4957 bytes) taskResourceAssignments Map() 2021-06-28 09:33:12 Executor: INFO: Running task 1.0 in stage 0.0 (TID 1) 2021-06-28 09:33:12 root: INFO: RegionPool: initialized for thread 63: Executor task launch worker for task 1.0 in stage 0.0 (TID 1) 2021-06-28 09:33:12 TaskSetManager: INFO: Finished task 0.0 in stage 0.0 (TID 0) in 9230835 ms on holy7c02601.rc.fas.harvard.edu (executor driver) (1/481) 2021-06-28 09:33:12 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), regions.size = 3, 0 current java objects, thread 63: Executor task launch worker for task 1.0 in stage 0.0 (TID 1) 2021-06-28 09:33:12 root: INFO: RegionPool: REPORT_THRESHOLD: 544.0K allocated (320.0K blocks / 224.0K chunks), regions.size = 4, 0 current java objects, thread 63: Executor task launch worker for task 1.0 in stage 0.0 (TID 1) 2021-06-28 09:33:12 root: INFO: RegionPool: REPORT_THRESHOLD: 1.0M allocated (448.0K blocks / 625.0K chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 1.0 in stage 0.0 (TID 1) 2021-06-28 09:33:12 root: INFO: RegionPool: REPORT_THRESHOLD: 2.0M allocated (1.0M blocks / 1.0M chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 1.0 in stage 0.0 (TID 1) 2021-06-28 09:33:12 root: INFO: RegionPool: REPORT_THRESHOLD: 7.1M allocated (2.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 1.0 in stage 0.0 (TID 1) 2021-06-28 10:16:14 root: INFO: RegionPool: REPORT_THRESHOLD: 8.0M allocated (3.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 1.0 in stage 0.0 (TID 1) 2021-06-28 12:18:48 root: INFO: Index reader cache queries: 18 2021-06-28 12:18:48 root: INFO: Index reader cache hit rate: 0.4444444444444444 2021-06-28 12:18:48 root: INFO: TaskReport: stage=0, partition=1, attempt=0, peakBytes=10579368, peakBytesReadable=10.09 MiB 2021-06-28 12:18:48 root: INFO: RegionPool: FREE: 5.3M allocated (5.3M blocks / 0 chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 1.0 in stage 0.0 (TID 1) 2021-06-28 12:18:48 Executor: INFO: Finished task 1.0 in stage 0.0 (TID 1). 1048 bytes result sent to driver 2021-06-28 12:18:48 TaskSetManager: INFO: Starting task 2.0 in stage 0.0 (TID 2) (holy7c02601.rc.fas.harvard.edu, executor driver, partition 2, PROCESS_LOCAL, 4957 bytes) taskResourceAssignments Map() 2021-06-28 12:18:48 Executor: INFO: Running task 2.0 in stage 0.0 (TID 2) 2021-06-28 12:18:48 root: INFO: RegionPool: initialized for thread 63: Executor task launch worker for task 2.0 in stage 0.0 (TID 2) 2021-06-28 12:18:48 TaskSetManager: INFO: Finished task 1.0 in stage 0.0 (TID 1) in 9936011 ms on holy7c02601.rc.fas.harvard.edu (executor driver) (2/481) 2021-06-28 12:20:28 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), regions.size = 3, 0 current java objects, thread 63: Executor task launch worker for task 2.0 in stage 0.0 (TID 2) 2021-06-28 12:20:28 root: INFO: RegionPool: REPORT_THRESHOLD: 544.0K allocated (320.0K blocks / 224.0K chunks), regions.size = 4, 0 current java objects, thread 63: Executor task launch worker for task 2.0 in stage 0.0 (TID 2) 2021-06-28 12:20:28 root: INFO: RegionPool: REPORT_THRESHOLD: 1.0M allocated (448.0K blocks / 625.0K chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 2.0 in stage 0.0 (TID 2) 2021-06-28 12:20:28 root: INFO: RegionPool: REPORT_THRESHOLD: 2.0M allocated (1.0M blocks / 1.0M chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 2.0 in stage 0.0 (TID 2) 2021-06-28 12:20:28 root: INFO: RegionPool: REPORT_THRESHOLD: 7.1M allocated (2.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 2.0 in stage 0.0 (TID 2) 2021-06-28 12:56:26 root: INFO: RegionPool: REPORT_THRESHOLD: 8.0M allocated (3.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 2.0 in stage 0.0 (TID 2) 2021-06-28 15:03:29 root: INFO: Index reader cache queries: 16 2021-06-28 15:03:29 root: INFO: Index reader cache hit rate: 0.4375 2021-06-28 15:03:29 root: INFO: TaskReport: stage=0, partition=2, attempt=0, peakBytes=10448296, peakBytesReadable=9.96 MiB 2021-06-28 15:03:29 root: INFO: RegionPool: FREE: 5.2M allocated (5.2M blocks / 0 chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 2.0 in stage 0.0 (TID 2) 2021-06-28 15:03:29 Executor: INFO: Finished task 2.0 in stage 0.0 (TID 2). 1048 bytes result sent to driver 2021-06-28 15:03:29 TaskSetManager: INFO: Starting task 3.0 in stage 0.0 (TID 3) (holy7c02601.rc.fas.harvard.edu, executor driver, partition 3, PROCESS_LOCAL, 4957 bytes) taskResourceAssignments Map() 2021-06-28 15:03:29 Executor: INFO: Running task 3.0 in stage 0.0 (TID 3) 2021-06-28 15:03:29 root: INFO: RegionPool: initialized for thread 63: Executor task launch worker for task 3.0 in stage 0.0 (TID 3) 2021-06-28 15:03:29 TaskSetManager: INFO: Finished task 2.0 in stage 0.0 (TID 2) in 9880495 ms on holy7c02601.rc.fas.harvard.edu (executor driver) (3/481) 2021-06-28 15:03:29 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), regions.size = 3, 0 current java objects, thread 63: Executor task launch worker for task 3.0 in stage 0.0 (TID 3) 2021-06-28 15:03:29 root: INFO: RegionPool: REPORT_THRESHOLD: 544.0K allocated (320.0K blocks / 224.0K chunks), regions.size = 4, 0 current java objects, thread 63: Executor task launch worker for task 3.0 in stage 0.0 (TID 3) 2021-06-28 15:03:29 root: INFO: RegionPool: REPORT_THRESHOLD: 1.0M allocated (448.0K blocks / 625.0K chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 3.0 in stage 0.0 (TID 3) 2021-06-28 15:03:29 root: INFO: RegionPool: REPORT_THRESHOLD: 2.0M allocated (1.0M blocks / 1.0M chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 3.0 in stage 0.0 (TID 3) 2021-06-28 15:03:29 root: INFO: RegionPool: REPORT_THRESHOLD: 7.1M allocated (2.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 3.0 in stage 0.0 (TID 3) 2021-06-28 15:48:20 root: INFO: RegionPool: REPORT_THRESHOLD: 8.0M allocated (3.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 3.0 in stage 0.0 (TID 3) 2021-06-28 17:55:12 root: INFO: Index reader cache queries: 18 2021-06-28 17:55:12 root: INFO: Index reader cache hit rate: 0.4444444444444444 2021-06-28 17:55:12 root: INFO: TaskReport: stage=0, partition=3, attempt=0, peakBytes=10513832, peakBytesReadable=10.03 MiB 2021-06-28 17:55:12 root: INFO: RegionPool: FREE: 5.3M allocated (5.3M blocks / 0 chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 3.0 in stage 0.0 (TID 3) 2021-06-28 17:55:13 Executor: INFO: Finished task 3.0 in stage 0.0 (TID 3). 1091 bytes result sent to driver 2021-06-28 17:55:13 TaskSetManager: INFO: Starting task 4.0 in stage 0.0 (TID 4) (holy7c02601.rc.fas.harvard.edu, executor driver, partition 4, PROCESS_LOCAL, 4957 bytes) taskResourceAssignments Map() 2021-06-28 17:55:13 Executor: INFO: Running task 4.0 in stage 0.0 (TID 4) 2021-06-28 17:55:13 root: INFO: RegionPool: initialized for thread 63: Executor task launch worker for task 4.0 in stage 0.0 (TID 4) 2021-06-28 17:55:13 TaskSetManager: INFO: Finished task 3.0 in stage 0.0 (TID 3) in 10303844 ms on holy7c02601.rc.fas.harvard.edu (executor driver) (4/481) 2021-06-28 17:55:13 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), regions.size = 3, 0 current java objects, thread 63: Executor task launch worker for task 4.0 in stage 0.0 (TID 4) 2021-06-28 17:55:13 root: INFO: RegionPool: REPORT_THRESHOLD: 544.0K allocated (320.0K blocks / 224.0K chunks), regions.size = 4, 0 current java objects, thread 63: Executor task launch worker for task 4.0 in stage 0.0 (TID 4) 2021-06-28 17:55:13 root: INFO: RegionPool: REPORT_THRESHOLD: 1.0M allocated (448.0K blocks / 625.0K chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 4.0 in stage 0.0 (TID 4) 2021-06-28 17:55:13 root: INFO: RegionPool: REPORT_THRESHOLD: 2.0M allocated (1.0M blocks / 1.0M chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 4.0 in stage 0.0 (TID 4) 2021-06-28 17:55:13 root: INFO: RegionPool: REPORT_THRESHOLD: 7.1M allocated (2.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 4.0 in stage 0.0 (TID 4) 2021-06-28 18:39:03 root: INFO: RegionPool: REPORT_THRESHOLD: 8.0M allocated (3.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 4.0 in stage 0.0 (TID 4) 2021-06-28 20:46:51 root: INFO: Index reader cache queries: 18 2021-06-28 20:46:51 root: INFO: Index reader cache hit rate: 0.4444444444444444 2021-06-28 20:46:51 root: INFO: TaskReport: stage=0, partition=4, attempt=0, peakBytes=10579368, peakBytesReadable=10.09 MiB 2021-06-28 20:46:51 root: INFO: RegionPool: FREE: 5.3M allocated (5.3M blocks / 0 chunks), regions.size = 6, 0 current java objects, thread 63: Executor task launch worker for task 4.0 in stage 0.0 (TID 4) 2021-06-28 20:46:51 Executor: INFO: Finished task 4.0 in stage 0.0 (TID 4). 1048 bytes result sent to driver 2021-06-28 20:46:51 TaskSetManager: INFO: Starting task 5.0 in stage 0.0 (TID 5) (holy7c02601.rc.fas.harvard.edu, executor driver, partition 5, PROCESS_LOCAL, 4957 bytes) taskResourceAssignments Map() 2021-06-28 20:46:51 TaskSetManager: INFO: Finished task 4.0 in stage 0.0 (TID 4) in 10298400 ms on holy7c02601.rc.fas.harvard.edu (executor driver) (5/481) 2021-06-28 20:46:51 Executor: INFO: Running task 5.0 in stage 0.0 (TID 5) 2021-06-28 20:46:51 root: INFO: RegionPool: initialized for thread 119: Executor task launch worker for task 5.0 in stage 0.0 (TID 5) 2021-06-28 20:46:51 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), regions.size = 3, 0 current java objects, thread 119: Executor task launch worker for task 5.0 in stage 0.0 (TID 5) 2021-06-28 20:46:51 root: INFO: RegionPool: REPORT_THRESHOLD: 544.0K allocated (320.0K blocks / 224.0K chunks), regions.size = 4, 0 current java objects, thread 119: Executor task launch worker for task 5.0 in stage 0.0 (TID 5) 2021-06-28 20:46:51 root: INFO: RegionPool: REPORT_THRESHOLD: 1.0M allocated (448.0K blocks / 625.0K chunks), regions.size = 6, 0 current java objects, thread 119: Executor task launch worker for task 5.0 in stage 0.0 (TID 5) 2021-06-28 20:46:51 root: INFO: RegionPool: REPORT_THRESHOLD: 2.0M allocated (1.0M blocks / 1.0M chunks), regions.size = 6, 0 current java objects, thread 119: Executor task launch worker for task 5.0 in stage 0.0 (TID 5) 2021-06-28 20:46:51 root: INFO: RegionPool: REPORT_THRESHOLD: 7.1M allocated (2.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 119: Executor task launch worker for task 5.0 in stage 0.0 (TID 5) 2021-06-28 21:24:58 root: INFO: RegionPool: REPORT_THRESHOLD: 8.0M allocated (3.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 119: Executor task launch worker for task 5.0 in stage 0.0 (TID 5) 2021-06-28 23:39:09 root: INFO: Index reader cache queries: 16 2021-06-28 23:39:09 root: INFO: Index reader cache hit rate: 0.4375 2021-06-28 23:39:09 root: INFO: TaskReport: stage=0, partition=5, attempt=0, peakBytes=10448296, peakBytesReadable=9.96 MiB 2021-06-28 23:39:09 root: INFO: RegionPool: FREE: 5.2M allocated (5.2M blocks / 0 chunks), regions.size = 6, 0 current java objects, thread 119: Executor task launch worker for task 5.0 in stage 0.0 (TID 5) 2021-06-28 23:39:09 Executor: INFO: Finished task 5.0 in stage 0.0 (TID 5). 1048 bytes result sent to driver 2021-06-28 23:39:09 TaskSetManager: INFO: Starting task 6.0 in stage 0.0 (TID 6) (holy7c02601.rc.fas.harvard.edu, executor driver, partition 6, PROCESS_LOCAL, 4957 bytes) taskResourceAssignments Map() 2021-06-28 23:39:09 TaskSetManager: INFO: Finished task 5.0 in stage 0.0 (TID 5) in 10337720 ms on holy7c02601.rc.fas.harvard.edu (executor driver) (6/481) 2021-06-28 23:39:09 Executor: INFO: Running task 6.0 in stage 0.0 (TID 6) 2021-06-28 23:39:09 root: INFO: RegionPool: initialized for thread 130: Executor task launch worker for task 6.0 in stage 0.0 (TID 6) 2021-06-28 23:40:49 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), regions.size = 3, 0 current java objects, thread 130: Executor task launch worker for task 6.0 in stage 0.0 (TID 6) 2021-06-28 23:40:49 root: INFO: RegionPool: REPORT_THRESHOLD: 544.0K allocated (320.0K blocks / 224.0K chunks), regions.size = 4, 0 current java objects, thread 130: Executor task launch worker for task 6.0 in stage 0.0 (TID 6) 2021-06-28 23:40:49 root: INFO: RegionPool: REPORT_THRESHOLD: 1.0M allocated (448.0K blocks / 625.0K chunks), regions.size = 6, 0 current java objects, thread 130: Executor task launch worker for task 6.0 in stage 0.0 (TID 6) 2021-06-28 23:40:49 root: INFO: RegionPool: REPORT_THRESHOLD: 2.0M allocated (1.0M blocks / 1.0M chunks), regions.size = 6, 0 current java objects, thread 130: Executor task launch worker for task 6.0 in stage 0.0 (TID 6) 2021-06-28 23:40:49 root: INFO: RegionPool: REPORT_THRESHOLD: 7.1M allocated (2.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 130: Executor task launch worker for task 6.0 in stage 0.0 (TID 6) 2021-06-29 00:25:54 root: INFO: RegionPool: REPORT_THRESHOLD: 8.0M allocated (3.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 130: Executor task launch worker for task 6.0 in stage 0.0 (TID 6) 2021-06-29 02:33:13 root: INFO: Index reader cache queries: 18 2021-06-29 02:33:13 root: INFO: Index reader cache hit rate: 0.4444444444444444 2021-06-29 02:33:13 root: INFO: TaskReport: stage=0, partition=6, attempt=0, peakBytes=10513832, peakBytesReadable=10.03 MiB 2021-06-29 02:33:13 root: INFO: RegionPool: FREE: 5.3M allocated (5.3M blocks / 0 chunks), regions.size = 6, 0 current java objects, thread 130: Executor task launch worker for task 6.0 in stage 0.0 (TID 6) 2021-06-29 02:33:13 Executor: INFO: Finished task 6.0 in stage 0.0 (TID 6). 1048 bytes result sent to driver 2021-06-29 02:33:13 TaskSetManager: INFO: Starting task 7.0 in stage 0.0 (TID 7) (holy7c02601.rc.fas.harvard.edu, executor driver, partition 7, PROCESS_LOCAL, 4957 bytes) taskResourceAssignments Map() 2021-06-29 02:33:13 TaskSetManager: INFO: Finished task 6.0 in stage 0.0 (TID 6) in 10444123 ms on holy7c02601.rc.fas.harvard.edu (executor driver) (7/481) 2021-06-29 02:33:13 Executor: INFO: Running task 7.0 in stage 0.0 (TID 7) 2021-06-29 02:33:13 root: INFO: RegionPool: initialized for thread 142: Executor task launch worker for task 7.0 in stage 0.0 (TID 7) 2021-06-29 02:33:13 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), regions.size = 3, 0 current java objects, thread 142: Executor task launch worker for task 7.0 in stage 0.0 (TID 7) 2021-06-29 02:33:13 root: INFO: RegionPool: REPORT_THRESHOLD: 544.0K allocated (320.0K blocks / 224.0K chunks), regions.size = 4, 0 current java objects, thread 142: Executor task launch worker for task 7.0 in stage 0.0 (TID 7) 2021-06-29 02:33:13 root: INFO: RegionPool: REPORT_THRESHOLD: 1.0M allocated (448.0K blocks / 625.0K chunks), regions.size = 6, 0 current java objects, thread 142: Executor task launch worker for task 7.0 in stage 0.0 (TID 7) 2021-06-29 02:33:13 root: INFO: RegionPool: REPORT_THRESHOLD: 2.0M allocated (1.0M blocks / 1.0M chunks), regions.size = 6, 0 current java objects, thread 142: Executor task launch worker for task 7.0 in stage 0.0 (TID 7) 2021-06-29 02:33:13 root: INFO: RegionPool: REPORT_THRESHOLD: 7.1M allocated (2.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 142: Executor task launch worker for task 7.0 in stage 0.0 (TID 7) 2021-06-29 03:15:19 root: INFO: RegionPool: REPORT_THRESHOLD: 8.0M allocated (3.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 142: Executor task launch worker for task 7.0 in stage 0.0 (TID 7) 2021-06-29 05:14:25 root: INFO: Index reader cache queries: 18 2021-06-29 05:14:25 root: INFO: Index reader cache hit rate: 0.4444444444444444 2021-06-29 05:14:25 root: INFO: TaskReport: stage=0, partition=7, attempt=0, peakBytes=10579368, peakBytesReadable=10.09 MiB 2021-06-29 05:14:25 root: INFO: RegionPool: FREE: 5.3M allocated (5.3M blocks / 0 chunks), regions.size = 6, 0 current java objects, thread 142: Executor task launch worker for task 7.0 in stage 0.0 (TID 7) 2021-06-29 05:14:25 Executor: INFO: Finished task 7.0 in stage 0.0 (TID 7). 1048 bytes result sent to driver 2021-06-29 05:14:25 TaskSetManager: INFO: Starting task 8.0 in stage 0.0 (TID 8) (holy7c02601.rc.fas.harvard.edu, executor driver, partition 8, PROCESS_LOCAL, 4957 bytes) taskResourceAssignments Map() 2021-06-29 05:14:25 TaskSetManager: INFO: Finished task 7.0 in stage 0.0 (TID 7) in 9672257 ms on holy7c02601.rc.fas.harvard.edu (executor driver) (8/481) 2021-06-29 05:14:25 Executor: INFO: Running task 8.0 in stage 0.0 (TID 8) 2021-06-29 05:14:25 root: INFO: RegionPool: initialized for thread 153: Executor task launch worker for task 8.0 in stage 0.0 (TID 8) 2021-06-29 05:14:25 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), regions.size = 3, 0 current java objects, thread 153: Executor task launch worker for task 8.0 in stage 0.0 (TID 8) 2021-06-29 05:14:25 root: INFO: RegionPool: REPORT_THRESHOLD: 544.0K allocated (320.0K blocks / 224.0K chunks), regions.size = 4, 0 current java objects, thread 153: Executor task launch worker for task 8.0 in stage 0.0 (TID 8) 2021-06-29 05:14:25 root: INFO: RegionPool: REPORT_THRESHOLD: 1.0M allocated (448.0K blocks / 625.0K chunks), regions.size = 6, 0 current java objects, thread 153: Executor task launch worker for task 8.0 in stage 0.0 (TID 8) 2021-06-29 05:14:25 root: INFO: RegionPool: REPORT_THRESHOLD: 2.0M allocated (1.0M blocks / 1.0M chunks), regions.size = 6, 0 current java objects, thread 153: Executor task launch worker for task 8.0 in stage 0.0 (TID 8) 2021-06-29 05:14:25 root: INFO: RegionPool: REPORT_THRESHOLD: 7.1M allocated (2.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 153: Executor task launch worker for task 8.0 in stage 0.0 (TID 8) 2021-06-29 05:49:07 root: INFO: RegionPool: REPORT_THRESHOLD: 8.0M allocated (3.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 153: Executor task launch worker for task 8.0 in stage 0.0 (TID 8) 2021-06-29 08:01:49 root: INFO: Index reader cache queries: 16 2021-06-29 08:01:49 root: INFO: Index reader cache hit rate: 0.4375 2021-06-29 08:01:49 root: INFO: TaskReport: stage=0, partition=8, attempt=0, peakBytes=10448296, peakBytesReadable=9.96 MiB 2021-06-29 08:01:49 root: INFO: RegionPool: FREE: 5.2M allocated (5.2M blocks / 0 chunks), regions.size = 6, 0 current java objects, thread 153: Executor task launch worker for task 8.0 in stage 0.0 (TID 8) 2021-06-29 08:01:49 Executor: INFO: Finished task 8.0 in stage 0.0 (TID 8). 1048 bytes result sent to driver 2021-06-29 08:01:49 TaskSetManager: INFO: Starting task 9.0 in stage 0.0 (TID 9) (holy7c02601.rc.fas.harvard.edu, executor driver, partition 9, PROCESS_LOCAL, 4957 bytes) taskResourceAssignments Map() 2021-06-29 08:01:49 TaskSetManager: INFO: Finished task 8.0 in stage 0.0 (TID 8) in 10044295 ms on holy7c02601.rc.fas.harvard.edu (executor driver) (9/481) 2021-06-29 08:01:49 Executor: INFO: Running task 9.0 in stage 0.0 (TID 9) 2021-06-29 08:01:49 root: INFO: RegionPool: initialized for thread 164: Executor task launch worker for task 9.0 in stage 0.0 (TID 9) 2021-06-29 08:01:49 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), regions.size = 3, 0 current java objects, thread 164: Executor task launch worker for task 9.0 in stage 0.0 (TID 9) 2021-06-29 08:01:49 root: INFO: RegionPool: REPORT_THRESHOLD: 544.0K allocated (320.0K blocks / 224.0K chunks), regions.size = 4, 0 current java objects, thread 164: Executor task launch worker for task 9.0 in stage 0.0 (TID 9) 2021-06-29 08:01:49 root: INFO: RegionPool: REPORT_THRESHOLD: 1.0M allocated (448.0K blocks / 625.0K chunks), regions.size = 6, 0 current java objects, thread 164: Executor task launch worker for task 9.0 in stage 0.0 (TID 9) 2021-06-29 08:01:49 root: INFO: RegionPool: REPORT_THRESHOLD: 2.0M allocated (1.0M blocks / 1.0M chunks), regions.size = 6, 0 current java objects, thread 164: Executor task launch worker for task 9.0 in stage 0.0 (TID 9) 2021-06-29 08:01:49 root: INFO: RegionPool: REPORT_THRESHOLD: 7.1M allocated (2.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 164: Executor task launch worker for task 9.0 in stage 0.0 (TID 9) 2021-06-29 08:44:02 root: INFO: RegionPool: REPORT_THRESHOLD: 8.0M allocated (3.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 164: Executor task launch worker for task 9.0 in stage 0.0 (TID 9) 2021-06-29 10:43:36 root: INFO: Index reader cache queries: 18 2021-06-29 10:43:36 root: INFO: Index reader cache hit rate: 0.4444444444444444 2021-06-29 10:43:36 root: INFO: TaskReport: stage=0, partition=9, attempt=0, peakBytes=10513832, peakBytesReadable=10.03 MiB 2021-06-29 10:43:36 root: INFO: RegionPool: FREE: 5.3M allocated (5.3M blocks / 0 chunks), regions.size = 6, 0 current java objects, thread 164: Executor task launch worker for task 9.0 in stage 0.0 (TID 9) 2021-06-29 10:43:36 Executor: INFO: Finished task 9.0 in stage 0.0 (TID 9). 1048 bytes result sent to driver 2021-06-29 10:43:36 TaskSetManager: INFO: Starting task 10.0 in stage 0.0 (TID 10) (holy7c02601.rc.fas.harvard.edu, executor driver, partition 10, PROCESS_LOCAL, 4957 bytes) taskResourceAssignments Map() 2021-06-29 10:43:36 TaskSetManager: INFO: Finished task 9.0 in stage 0.0 (TID 9) in 9706791 ms on holy7c02601.rc.fas.harvard.edu (executor driver) (10/481) 2021-06-29 10:43:36 Executor: INFO: Running task 10.0 in stage 0.0 (TID 10) 2021-06-29 10:43:36 root: INFO: RegionPool: initialized for thread 175: Executor task launch worker for task 10.0 in stage 0.0 (TID 10) 2021-06-29 10:43:36 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), regions.size = 3, 0 current java objects, thread 175: Executor task launch worker for task 10.0 in stage 0.0 (TID 10) 2021-06-29 10:43:36 root: INFO: RegionPool: REPORT_THRESHOLD: 544.0K allocated (320.0K blocks / 224.0K chunks), regions.size = 4, 0 current java objects, thread 175: Executor task launch worker for task 10.0 in stage 0.0 (TID 10) 2021-06-29 10:43:36 root: INFO: RegionPool: REPORT_THRESHOLD: 1.0M allocated (448.0K blocks / 625.0K chunks), regions.size = 6, 0 current java objects, thread 175: Executor task launch worker for task 10.0 in stage 0.0 (TID 10) 2021-06-29 10:43:36 root: INFO: RegionPool: REPORT_THRESHOLD: 2.0M allocated (1.0M blocks / 1.0M chunks), regions.size = 6, 0 current java objects, thread 175: Executor task launch worker for task 10.0 in stage 0.0 (TID 10) 2021-06-29 10:43:36 root: INFO: RegionPool: REPORT_THRESHOLD: 7.1M allocated (2.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 175: Executor task launch worker for task 10.0 in stage 0.0 (TID 10) 2021-06-29 11:24:41 root: INFO: RegionPool: REPORT_THRESHOLD: 8.0M allocated (3.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 175: Executor task launch worker for task 10.0 in stage 0.0 (TID 10) 2021-06-29 13:29:55 root: INFO: Index reader cache queries: 18 2021-06-29 13:29:55 root: INFO: Index reader cache hit rate: 0.4444444444444444 2021-06-29 13:29:55 root: INFO: TaskReport: stage=0, partition=10, attempt=0, peakBytes=10579368, peakBytesReadable=10.09 MiB 2021-06-29 13:29:55 root: INFO: RegionPool: FREE: 5.3M allocated (5.3M blocks / 0 chunks), regions.size = 6, 0 current java objects, thread 175: Executor task launch worker for task 10.0 in stage 0.0 (TID 10) 2021-06-29 13:29:55 Executor: INFO: Finished task 10.0 in stage 0.0 (TID 10). 1092 bytes result sent to driver 2021-06-29 13:29:55 TaskSetManager: INFO: Starting task 11.0 in stage 0.0 (TID 11) (holy7c02601.rc.fas.harvard.edu, executor driver, partition 11, PROCESS_LOCAL, 4957 bytes) taskResourceAssignments Map() 2021-06-29 13:29:55 TaskSetManager: INFO: Finished task 10.0 in stage 0.0 (TID 10) in 9979171 ms on holy7c02601.rc.fas.harvard.edu (executor driver) (11/481) 2021-06-29 13:29:55 Executor: INFO: Running task 11.0 in stage 0.0 (TID 11) 2021-06-29 13:29:55 root: INFO: RegionPool: initialized for thread 186: Executor task launch worker for task 11.0 in stage 0.0 (TID 11) 2021-06-29 13:31:35 root: INFO: RegionPool: REPORT_THRESHOLD: 256.0K allocated (256.0K blocks / 0 chunks), regions.size = 3, 0 current java objects, thread 186: Executor task launch worker for task 11.0 in stage 0.0 (TID 11) 2021-06-29 13:31:35 root: INFO: RegionPool: REPORT_THRESHOLD: 544.0K allocated (320.0K blocks / 224.0K chunks), regions.size = 4, 0 current java objects, thread 186: Executor task launch worker for task 11.0 in stage 0.0 (TID 11) 2021-06-29 13:31:35 root: INFO: RegionPool: REPORT_THRESHOLD: 1.0M allocated (448.0K blocks / 625.0K chunks), regions.size = 6, 0 current java objects, thread 186: Executor task launch worker for task 11.0 in stage 0.0 (TID 11) 2021-06-29 13:31:35 root: INFO: RegionPool: REPORT_THRESHOLD: 2.0M allocated (1.0M blocks / 1.0M chunks), regions.size = 6, 0 current java objects, thread 186: Executor task launch worker for task 11.0 in stage 0.0 (TID 11) 2021-06-29 13:31:35 root: INFO: RegionPool: REPORT_THRESHOLD: 7.1M allocated (2.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 186: Executor task launch worker for task 11.0 in stage 0.0 (TID 11) 2021-06-29 14:10:41 root: INFO: RegionPool: REPORT_THRESHOLD: 8.0M allocated (3.3M blocks / 4.8M chunks), regions.size = 6, 0 current java objects, thread 186: Executor task launch worker for task 11.0 in stage 0.0 (TID 11)