Irving Abrams moved with his family to Greenville, South Carolina, in 1936, where his father, Harry, led the effort to revive Temple of Israel, the city's Reform congregation. Harry managed the Piedmont Shirt Company, and hired African-Americans as early as 1939. Irving married Marjorie Kohler of Knoxville, Tennessee, followed his father into textiles, and oversaw the integration of his factory during the Civil Rights Movement.