- First rooted android device - HTC Dream(G1)
- First custom ROM - JesusFreke aka. JF ROM (Wasn't the First custom ROM but was most popular after G1 was rooted.)
Explanation:HTC Dream was the first commercially launched Android powered smartphone back in October 2008. Developers found exploit that gained them superuser access. It can be confirmed that HTC Dream or HTC G1 was the first rooted android device on it's Wikipedia page under the modding section.
JF wasn't the first. I couldn't find any source stating it was the first. I went back into the past using Google search tools and set the time from October 2008 to Jan 2009. What I found is an Interview with JesusFreke developer of JF Rom. Where it's stated that it was most popular ROM after G1 was rooted. Also this XDA forum post.