# Prepare hail entries for spark.ml

**URL:** <https://discuss.hail.is/t/prepare-hail-entries-for-spark-ml/1151>\
**Category:** Hail Query & hailctl\
**Created:** [October 29, 2019, 9:03am UTC](https://discuss.hail.is/t/prepare-hail-entries-for-spark-ml/1151 "2019-10-29T09:03:43Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hominidae](https://avatars.discourse-cdn.com/v4/letter/h/258eb7/32.png) [@Hominidae](https://discuss.hail.is/u/Hominidae)\
**Post date:** [October 29, 2019, 9:03am UTC](https://discuss.hail.is/t/prepare-hail-entries-for-spark-ml/1151/1 "2019-10-29T09:03:43Z")

</div>

Hi,

I’m trying to pull entries data from hail to build predictive model with spark.ml. I came up with the following codes:

```
mt = hl.read_matrix_table(tmp)
mt = mt.unfilter_entries()
mt = mt.annotate_cols(g = hl.agg.collect(mt.GT.n_alt_alleles()))
test = mt.cols().select('g').to_spark()

```

and this is what I got:  
 ![image](https://canada1.discourse-cdn.com/flex036/uploads/hail/original/1X/c642e13f5d79448c124a99b02a71ffc417bceaa2.png)

The problem of this snippet is that hl.agg.collect() doesn’t guarantee the order of the array, which makes the feature tracking hard. Anyone has a solution for this? Would hl.str(mt.locus).collect() work?

Thanks a lot!

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [October 29, 2019, 12:03pm UTC](https://discuss.hail.is/t/prepare-hail-entries-for-spark-ml/1151/2 "2019-10-29T12:03:39Z")

</div>

The order of hl.agg.collect()'s result here is the ordering of the `tmp` matrix table (at least it should be)

---

<div class="post-metadata">

**Author:** ![Hominidae](https://avatars.discourse-cdn.com/v4/letter/h/258eb7/32.png) [@Hominidae](https://discuss.hail.is/u/Hominidae)\
**Post date:** [October 29, 2019, 12:18pm UTC](https://discuss.hail.is/t/prepare-hail-entries-for-spark-ml/1151/3 "2019-10-29T12:18:42Z")

</div>

Great, thanks!

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [October 29, 2019, 12:22pm UTC](https://discuss.hail.is/t/prepare-hail-entries-for-spark-ml/1151/4 "2019-10-29T12:22:59Z")

</div>

ah, we need to fix the docs!