"You're 29 years old and what have you done with your life?" his sister snapped as the pair squabbled over his request, her brother later recalled. Edwards assented, but only after Gordy signed a contract pledging future royalties as security.
Stevie Wonder and managed the careers of such era-defining artists as Smokey Robinson, Marvin Gaye and the Supremes.
Yet Edwards made her most enduring mark after the company moved to Los Angeles in 1972. She stayed behind and in 1985 turned the original offices known as Hitsville USA into the Motown Historical Museum. Then she packed it with the artifacts she had pointedly saved along the way.
Edwards died Wednesday in Detroit of natural causes, the museum announced. She was 91.
"Whatever she did, it was with the highest standards," her brother said in a statement. "She preserved Motown memorabilia before it was memorabilia, collecting our history long before we knew we were making it."
Wonder said he was "taken back by the loss" of Edwards, whom he regarded as "another mother."
When Wonder came to Motown as a boy, Edwards helped him manage his money, arranged for tutors and enrolled him at the Michigan School of the Blind.
"She believed in me — when I was 14 years old," Wonder said in a statement. "She championed me being in Motown. I shared with her many of my songs first before anyone else."
Edwards played a key role managing young acts in the 1960s. Eventually she rose to vice president and directed Motown's international operations.
"Poor kids from broken homes would rush here after school and hang out all night," Edwards said of Hitsville in a 1989 Times article. "Between 1959 and 1972, this little house was like home for a lot of kids. Without Motown, most of the talent discovered in this building would have been overlooked by society."
Edwards was "born bossy," her brother once said, on April 25, 1920, in Oconee, Ga. She was one of eight children of Berry and Bertha Gordy and as a toddler moved to Detroit with her family.
Esther attended both Howard University in Washington, D.C., and Wayne State University in Detroit.
She married in the early 1940s and had a son before divorcing.
In 1951, she married George H. Edwards, who went on to serve in the Michigan Legislature. He died in 1980.
She had helped set up the Gordy family savings club as a source of financing for business ventures when her brother sought the loan that led to the "Motown Sound."
Inside Motown, the 4-foot-10 Edwards was called a "pack rat," teased for squirreling away everything she could — concert posters, fliers, stage costumes and other would-be collectibles — during her 30 years with the company.
"We used to laugh at Mrs. Edwards because everywhere we went on those tours, she saved everything. She saved all the pictures, all the placards," Robinson told the Detroit Free Press in 2005. "But what a wonderful thing she did. Because of her we have that museum, we have that place where people can go and see that history."
More than 40,000 people visit each year, according to the museum. Exhibits have included girl-group gowns, record covers, Michael Jackson's sequined glove and the upstairs quarters where Berry once lived.
The need for such a museum dawned on Edwards over time as tourists dropped by the offices she kept at Hitsville after the company had moved west.
Once about 50 men sporting white sailor uniforms and British accents showed up, explaining that they had rented vans to drive the 60 miles to Motown after their ship had docked in Toledo, Ohio, she told Smithsonian magazine in 1994.
"That was the turning point," Edwards said. "I thought, 'Well, gosh, maybe we did make history here.'"
In addition to her brother, Berry, Edwards is survived by her son, Robert Berry Bullock; stepson Harry T. Edwards; two other siblings, Anna Gordy Gaye and Robert L. Gordy; three granddaughters; and six great-grandchildren.