Albert Edward Brumley was born on October 29, 1905, near Spiro, Oklahoma, into a family of sharecroppers. He grew up in a musically inclined household, where his father was a fiddler and his mother enjoyed singing. Despite the family's humble means, music played a significant role in their daily life, with gatherings often centered around musical performances. Brumley attended public school in Rock Island, Oklahoma, completing up to the tenth grade, and developed a passion for gospel music at a young age.

In 1926, Brumley enrolled at the Hartford Musical Institute in Hartford, Arkansas, under the mentorship of Eugene Monroe Bartlett, a renowned figure in gospel music. Bartlett's guidance was instrumental in shaping Brumley's career, and Brumley eventually became one of Bartlett's finest songwriters. He published his first song, "I Can Hear Them Singing Over There," in 1927 and gained prominence as a bass singer with the Hartford Quartet.

Brumley married Goldie Edith Schell on August 30, 1931, and they settled in Powell, Missouri, where they raised six children. He continued to write music, producing over 800 songs throughout his career. Some of his most notable compositions include "I'll Fly Away," "Turn Your Radio On," and "He Set Me Free." His work became widely popular, with "I'll Fly Away" being recorded by numerous artists across various genres.

In 1943, Brumley founded his own publishing company, Albert E. Brumley and Sons, and purchased the Hartford Music Company in 1948. He established the Albert E. Brumley Sundown to Sunup Gospel Sing in 1969, which continues to be a celebrated event.

Albert E. Brumley passed away on November 15, 1977, in Powell, Missouri, and was buried at the Fox Church of Christ Cemetery. His legacy endures through his contributions to gospel music, and he has been inducted into the Nashville Songwriters Hall of Fame, Gospel Music Hall of Fame, and Oklahoma Music Hall of Fame.