Flaherty, one of Canada's longest-serving finance ministers, resigned from cabinet in March. He had been Prime Minister Stephen Harper's only finance minister since the Conservatives were elected to power in 2006.
Ottawa police confirmed Flaherty died of a heart attack early Thursday afternoon.
Flaherty and his wife, Ontario MPP Christine Elliott, have 22-year-old triplet sons — John, Galen, and Quinn.
The family released a statement Thursday afternoon saying Flaherty "passed away peacefully in Ottawa."
"We appreciate he was so well-supported in public life by Canadians from coast to coast to coast and by his international colleagues," it read.
Harper addressed his caucus later in the afternoon, calling Flaherty's passing "an unexpected and terrible shock to Jim's family, to our caucus, to Laureen and me."
The prime minister called Flaherty a partner and friend.
"It is with the heaviest of hearts that I offer my family's condolences," he said. "Christine, Quinn, Galen and John, you're in all of our thoughts and prayers."
The Ottawa Citizen reported that Labour Minister Kellie Leitch, a medical doctor, administered CPR to Flaherty before paramedics arrived at his condo.
Leitch and Flaherty lived in the same building, and she credited the former finance minister for encouraging her to enter politics. She was visibly upset during Harper's press conference about Flaherty's death.
After PMSH finished speech today, he and Laureen went straight to @KellieLeitch. Now I know why. She administered CPR to @JimFlaherty.
— Stephen Maher (@stphnmaher) April 10, 2014NDP Leader Thomas Mulcair fought back tears while addressing reporters.
"Catherine and I want to express to Christine Elliott our profound sadness at the departure of our friend Jim Flaherty. All his colleagues in the House of Commons share in that loss," Mulcair said.
"Jim Flaherty was an extraordinary, dedicated public servant. He will be greatly missed by all of us."
In a statement, Liberal Leader Justin Trudeau said Flaherty was a dedicated Parliamentarian with "a strong social conscience" who was committed to the ideal of public service.
"He was a proud Canadian and showed continued devotion and commitment to his country over many years," Trudeau said in the statement. "This is a loss to the entire family in the House of Commons."
Flaherty, who represented the Ontario riding of Whitby-Oshawa since 2006 after a long career in Ontario politics, had been struggling with a rare skin condition — bullous pemphigoid. The condition was not thought to be life-threatening.
When he stepped down on March 18, Flaherty said in a statement his health issues were not the reason for his resignation.
"I am happy to report that I am on the road to a full recovery and the decision to leave politics was not related in any way to my health," he said in a statement. "This decision was made because it is the right one for me and my family at this time."
The House of Commons abruptly suspended its sitting before question period began. The Peace Tower flag was lowered to haf-mast in his honour. The Ontario legislature, where Flaherty sat for a decade, also held a moment's silence for him and recessed.
Politicians of all stripes took to Twitter Thursday afternoon to share condolences.
Just crushed at the loss of my colleague and friend @JimFlaherty. We spent 25 years together in public life. An Irish lion is gone.
— Tony Clement (@TonyclementCPC) April 10, 2014Jim was a mentor to me at Queen's Park. I could rely on Jim to be a devout friend through tough times and an encouraging figure through good
— John Baird (@HonJohnBaird) April 10, 2014This spot, this place, won't ever be the same. Thank you Jim. pic.twitter.com/vac2tVOjZM
— James Moore (@JamesMoore_org) April 10, 2014My dear friend Jim Flaherty was a great man and a great Canadian. My heartfelt condolences to Christine, John, Galen & Quinn.
— Peter MacKay (@MinPeterMacKay) April 10, 2014Still can't quite believe that Jim Flaherty has left us. The lyrical Irish spirit, gritty & gregarious, the fighter, the loyal friend. RIP.
— Jason Kenney (@kenneyjason) April 10, 2014Devastating news. Jim Flaherty was a good person and a good friend. Condolences to family and friends.
— John Duncan (@JohnDuncanMP) April 10, 2014Like all Canadians, I join in expressing my sadness at Jim Flaherty’s passing. My sincere condolences to @chriselliottpc and his children.
— Justin Trudeau, MP (@JustinTrudeau) April 10, 2014I will always remember the kindness he showed me as a friend & new Minister. He was tenacious, effective, kind, and tough.
— Kevin Sorenson (@MinKSorenson) April 10, 2014Our hearts are breaking at the loss of a great colleague and friend. We will miss you Jim. My prayers are with Christine and the children.
— Candice Bergen (@CandiceBergenMP) April 10, 2014So saddened at news of Jim Flaherty's passing. My deepest sympathies to his family and friends. A genuinely likeable, widely respected man.
— Craig Scott (@CraigScottNDP) April 10, 2014Much sadness. Thoughts are with family of Jim Flaherty. He gave so much to public service in Canada
— Libby Davies (@LibbyDavies) April 10, 2014I'm saddened & shocked, words cannot describe the great loss to our country and community.
— Colin Carrie (@ColinCarrie) April 10, 2014My thoughts and prayers go out to the family of Jim Flaherty. Thank you for sharing Jim with us, and allowing him to serve Canada so well.
— Mark Strahl, MP (@markstrahl) April 10, 2014Shocking news. My wife Paula and I send our deepest condolences to Jim Flaherty's family, friends,and colleagues.
— Robert Chisholm (@RobertNDP) April 10, 2014Jim Flaherty's passing a tragic loss for his family and Canada: a fine person, a great political leader, he will be missed #cdnpoli
— Peter Kent, MP (@KentThornhillMP) April 10, 2014Very saddened to learn the news regarding the passing of former Finance Minister Jim Flaherty. His committment to Canada was unquestioned
— Glenn Thibeault (@GlennThibeault) April 10, 2014My heart goes out to the family of Jim Flaherty, & all my Conservative colleagues. Know there is love & support around you.
— Megan Leslie, MP (@MeganLeslieMP) April 10, 2014Very sad on passing of Jim Flaherty. We didn't always agree but got on well - he once called me "endangered species" as only Lib MP in 905!
— John McCallum (@JohnMcCallumMP) April 10, 2014Sincere condolences to Jim Flaherty’s family. The world of Canadian politics has lost a brilliant man.
— Hon. Stéphane Dion (@HonStephaneDion) April 10, 2014Shocked and saddened to learn of Jim Flaherty's passing. My condolences go out to his friends, family and colleagues.
— Jinny Sims (@jinnysims) April 10, 2014My condolences to #JimFlaherty's family. You deserved more time with him. He was widely respected and will be missed.
— Peggy Nash (@PeggyNashNDP) April 10, 2014Jim, we already miss you. My best thoughts and prayers are with your family and loved ones. SB pic.twitter.com/7RfVxEnDkV
— Steven Blaney (@MinStevenBlaney) April 10, 2014With files from The Canadian Press