After moving to California in 1964, Black had a chance encounter with bassist Roy Estrada while trading in some cymbals at a pawn shop; shortly thereafter the two formed The Soul Giants, an R&B combo that was eventually augmented by singer Ray Collins. The Soul Giants earned a living working as a covers band until its ranks were infiltrated later that year by guitarist Frank Zappa, who convinced them to start performing original material (his) and changed the name of the band to The Mothers (although a short period was spent as Captain Glasspack and His Magic Mufflers).
Two years of poverty and struggle followed, but in 1966 the band landed a deal with MGM, resulting in the legendary debut double-record Freak Out! (1966). The Mothers (who had by this time been forced by their label to change their name to The Mothers of Invention) quickly became one of the most influential bands of their time, both through their inventive studio material (Absolutely Free (1967), We're Only In It For The Money (1968), Uncle Meat (1969)) and their controversial (and interactive) live performances, most notably demonstrated during a residency at the Garrick Theater in New York (1967). Black himself became known to audiences as 'the Indian of the group' - a fact he announced from time to time on We're Only In It For The Money.
When Zappa dissolved the original Mothers in 1969, Jimmy Carl Black put together Geronimo Black, a project with which he released a self-titled album in 1972. He also returned to work with Zappa in 1971 as an actor (and vocalist) in the acutely peculiar film 200 Motels. For his acting debut, Black played the role of Lonsome Cowboy Burt: a drunken redneck intent on copulating with waitresses, and who refuses to accept the diabolical truth about Theodore Bikel. An unfortunate lull in the drummer's took place during the years that followed, and he returned to Anthony for a while to make donuts. An unrewarding tenure with The Valley Loboys was more or less his only musical output for the remainder of the decade.
Early in the 80s Black re-united with some of his fellow former Mothers to create The Grandmothers, a group that performed both original material and songs from their Zappa years. Through the auspices of Rhino Records the Grandmothers released two albums, Grandmothers (1981) and Lookin' Up Granny's Dress (1982), before their first break-up. The band's existence subsequently pursued a convoluted existence, falling apart and re-forming several times over the next two decades, and even (for a period in the early 2000s) operating as two distinct entities. During another career lull in the mid-80s, Black moved to Austin and started a house painting company with his friend Arthur Brown - an enterprise which was named The Gentlemen of Color. A collaborative album between the two, Brown, Black and Blue, later materialized in 1989. Black also formed the R&B outfit Jimmy Carl Black and The Mannish Boys around this time, before once again resurrecting The Grandmothers at the close of the decade.
In 1991 Black began an enduring collaborative relationship with left-of-center guitarist Eugene Chadbourne, joining him in a large ensemble (that also included ex-Mother (and sometime Grandmother) Don Preston) for a performance in Europe at the Moers New Music Jazz Festival. The success of the trip resulted in Black finally freeing himself of the dismal U.S. and moving with his wife to Italy. After this move, the drummer's musical activity kicked into high gear, and tours and records with Chadbourne, The Grandmothers, The Farrell and Black Band, and The Muffin Men materialized regularly throughout the 90s and 00s. In order to facilitate the release of his new material Black founded Inkanish records in 2001 - a label that initially just issued material on CDR, but which moved into regular production in 2002.