My question is: Do I have to get rid of all the interest that was added since the money was firstly put in the bank, or only the interest of the last one or two years before I cancelled it?.
It is not permissible to put money in riba-based banks except in cases of necessity in order to protect it when there is no Islamic bank. In that case one should only use a current account (i.e., without interest), on the basis of doing the lesser of two evils. The interest that is taken for depositing money in a riba-based bank is haraam and it is riba which has been forbidden by Allaah and His Messenger in emphatic terms. Hence you should get rid of it by spending it in the public interest and on charitable causes. See the answer to question no. 20876 and 45691.
This getting rid of it is obligatory for the one who dealt with riba. As for the heirs, they may benefit from this money according to some of the scholars, because that which it is haraam to acquire – such as riba – is haraam only for the one who acquires it, not for the one to whom the money is transferred by permissible means such as inheritance, a gift and so on.
Shaykh Ibn ‘Uthaymeen (may Allaah have mercy on him) said: As for that which is haraam because of the way in which it is acquired, it is haraam for the one who acquires it, such as riba. If the person who dealt with riba dies, then his wealth is permissible for his heirs. As for that which is haraam in and of itself, such as alcohol, it is haraam for the one who passes it on and the one to whom it is passed on. The same applies to that which is haraam and remains haraam, such as usurped and stolen property. If a person steals something then dies, it is not permissible for the heir. If he knows who the owner is he should give it to him, otherwise he should give it in charity on his behalf. End quote from Liqa’aat al-Baab il-Maftooh (1/304).
Based on this, there is nothing wrong with you benefiting from this wealth, but it should be noted that you have to get rid of the interest that was added to the money after your father died, because the money came into your possession on the death of your father. As for the interest that your father took during his lifetime, there is nothing wrong with you benefiting from it, and you do not have to get rid of it.
Some scholars are of the view that riba-based interest should not be included in the estate, and that one should get rid of it. This was stated in a fatwa by the scholars of the Standing Committee for Issuing Fatwas.
See: Fataawa al-Lajnah al-Daa’imah (16/455, 479).
But the former view is the one that we favour and follow.
And Allaah knows best.