SPOKANE, Wash. -

A federal case that could seriously impact Washington's new marijuana industry will be decided in Spokane as five people face decades in prison for growing medical pot on a remote property near Colville.

The defendants claim they were abiding by state law and only growing for themselves. However, the case is in federal court and under its laws marijuana is completely illegal. 

Many pot advocates worry if federal agents are going after medical marijuana patients, that the state licensed grows and pot shops could be next.

"We were growing a few pot plants up there just for our own consumption and along comes the law," defendant Larry Harvey said.

The medical marijuana grow of 74 plants was raided in August of 2012 at Harvey's home. Larry, his wife and three others face five marijuana related charges and a weapons charge. They are now called the 'Kettle Falls Five.' 

If all of them were growing cannabis individually, the 74 plants would be considered legal in Washington.

"So much so that the state declined to prosecute, the state declined to seize all of the marijuana," said Tony Serra, a high profile attorney from San Francisco working for the defendants.

Since law enforcement claims the five were growing the plants together, only 45 plants are allowed, which is why federal agents confiscated 29 plants, but left 45. They also seized several pounds of marijuana and pot-infused food.

National marijuana advocates question why prosecutors are moving forward with this case, while senior White House officials were quoted this week saying President Obama may pardon 'hundreds or thousands' of non-violent drug offenders.

"How, at the same time can they be planning to release people who've already been prosecuted and convicted and move forward with more prosecutions at the exact same people that they're trying to release?" said Kari Boiter from Americans for Safe Access.

Four of the defendants face up to 40 years in prison, while one faces life in prison if convicted.

Marijuana activists worry if the state chose not to prosecute these people, but the federal government did, what's to stop them from going after licensed grows and retail shops. Right now, more than a dozen people are licensed to grow recreational marijuana in Washington.

Even though the Department of Justice sent out a memo to U.S. Attorneys asking them to focus on drug traffickers and people who put pot into children's hands, the Assistant U.S. Attorney said in court Wednesday those are just guidelines they're not mandatory