Earlier this year, I served as a co-convener of the first-ever Bipartisan Summit on Criminal Justice Reform in Washington, D.C., with former White House adviser Van Jones, the American Conservative Union’s Pat Nolan and former House Speaker Newt Gingrich. Sponsors and partners included the American Civil Liberties Union and Drug Police Alliance on the left and the American Legislative Exchange Council and Koch Industries on the right.

This coalition came together to help lead a national conversation to reduce our bloated prison population, especially by freeing nonviolent offenders. Over the last 30 years, there has been a 500 percent increase in the number of inmates in federal prison, and nearly half of those are serving sentences for nonviolent drug offenses.

If you lived through the 1980s, you recall the dire, dystopian predictions of experts that we would soon face a world of “superpredators,” along with skyrocketing crime that would force families to flee to gated communities, with parents dashing to work and home like frightened gazelles.

It didn’t happen. Instead, crime has fallen steadily for the last 25 years, according to statistics compiled by the FBI and scholars. But while crime plunged 45 percent, our prison population skyrocketed.

Though the United States comprises just 5 percent of the world’s population, we have 25 percent of the world’s prison population. This makes us the world’s leading jailer, beating out countries like Russia, Iran, North Korea and China.

In addition, the costs of jailing are budget-busting. Taxpayers paid as much to house federal prisoners in 2013 as we paid to run the Department of Justice in 1980. Because of the jump in the number of federal prisoners, incarceration spending has increased by more than 1,100 percent over 30 years.

President Obama told the Bipartisan Summit in a video interview that a consequence of the ‘80s crackdown “was this massive trend towards incarceration, even of non-violent drug offenders,” who enter prison “at great expense to the state, many times train to become more hardened criminals while in prison, come out and are basically unemployable. And end up being pulled back in.”

Overall, 90 percent of the nation’s $260 billion spent on annual incarceration costs are borne at the local level. Van Jones, president of the Dream Corps and founder of #Cut50, stated, “Nearly 70 million Americans (have been) convicted of a criminal offense or served time in prison and still carry societal stigma as a result.”

Inimai Chettiar, director of New York University’s Brennan Center’s Justice Program, said, “We are not locking away the people who are habitual violent offenders anymore, we are locking up everyone.”

We have a prison population of 2.3 million Americans. Disproportionately, 60 to 75 percent of federal drug offenders are drawn from African-American and Latino youth, according to experts. This does not reflect the racial breakdown in drug offenses. What good does it do to jail people who pose no physical threat to others, including youth who need help to realize the promise of a future?

The Koch brothers want the release of nonviolent offenders speeded up. They propose ending law requirements that job applicants reveal past convictions. This produces nonviolent drug offenders who are jailed, then released into a community that will not employ them. Conservatives and liberals are united in the conviction that we need to give these youth second chances.

There is also a chance that Congress might act. Pending legislation in both the House and Senate would help reduce the number of nonviolent offenders in our jails. The Smarter Sentencing Act, offered by a bipartisan group of lawmakers, is an effort to give judges more leniency in sentencing nonviolent drug offenders on a case-by-case basis.

Under the proposed law, judges would be able to issue a sentence below the statutory mandatory minimum for a broader category of nonviolent drug offenses. The bill is designed to modernize federal judge sentencing policies in what could be an opportunity to weaken the status quo of continuing to overcrowd our prisons with nonviolent drug offenders.

President Obama is also likely to commute the sentences of 40-plus nonviolent drug possession offenders.

But it is only a drop in the bucket. More than 30,000 applications were received from prisoners seeking to qualify for the president’s clemency. His plan is largely symbolic, but it signals a turning point.

And perhaps at next year’s Bipartisan Summit on Criminal Justice Reform, we can build an even larger movement to make more Americans aware of the problems of our criminal justice system. We might even agree on several courses of action.

It’s time for those with the power to put changes in action to do so.

Donna Brazile is a senior Democratic strategist, a political commentator and contributor to CNN and ABC News, and a contributing columnist to Ms. Magazine and O, the Oprah Magazine.