Due to the directionality of transmissions in millimeter wave (mm-wave) networks, wireless stations are usually unable to overhear when other stations access the channel. This makes it hard to design efficient distributed beam coordination and scheduling mechanisms. At the same time, centralized schemes only perform well in relatively simple, static scenarios. In practical settings where links have different channel qualities and in the context of relaying or in-band backhauling, centrally coordinating all stations becomes difficult. In this paper, we propose a low complexity, decentralized, learning-based scheduling algorithm for mm-wave networks that handles heterogeneous link rates and packet sizes efficiently. Compared to state-of-theart slotted channel access for mm-wave networks, the proposed mechanism achieves throughput gains of up to a factor of 8 in single-hop scenarios and end-to-end throughput improvements of up to a factor of 1.6 in multi-hop topologies.