The objective of this thesis work is to investigate the 'softness' in nuclear incompressibility observed in the open shell nuclei and the role of symmetry energy in the nuclear incompressibility. The first experiment was performed to study isoscalar giant monopole resonance (ISGMR) strength distribution in the Cd isotopes. Accurate and extremely forward angle (including 0) alpha inelastic scattering measurements were made on a series of 106,110-116 Cd isotopes. The 'softness' in the nuclear incompressibility as observed in the Sn isotopes was seen in the Cd isotopes as well. The relativistic and non-relativistic calculations, which successfully reproduce the ISGMR strength distributions in 208Pb and 90Zr, over-estimate the ISGMR centroid energies in the Sn and the Cd isotopes. Inclusion of the various types of pairing effects in the calculation cannot account for the observed discrepancy. The second experiment was performed to test an intriguing idea of mutually enhanced magicity (MEM) effect playing a role in the nuclear incompressibility in order to account for the observed 'softness'. Systematic and extremely accurate alpha inelastic scattering measurements were made on 204,206,208Pb isotopes. The ISGMR centroid energies measured in the series of Pb isotopes indicated a standard A-1/3 dependence in stark contrast to a sharp increase of 0.6 MeV in the ISGMR centroid energy of 208Pb when compared to the centroid energy of 204Pb that was predicted as resulting from MEM effect. These results clearly established that the MEM effect does not play a measurable role in the energy of the ISGMR, thereby leaving the question of experimentally observed 'softness' in the Sn and Cd unanswered still. Paving a path for the future experiments, in order to answer the haunting question of experimentally observed 'softness' in nuclear incompressibility, the third experiment was performed to test the feasibility of using deuteron as a probe to measure ISGMR in radioactive isotopes using inverse kinematics. Accurate and extremely forward angle inelastic scattering measurements were made using high energy (100 MeV/u) deuteron beam. For the first time the multipole decomposition analysis was successfully employed to delineate different multipole contributions reliably. This experiment established the feasibility of using deuteron probe to study giant resonances in the radioactive nuclei and thus explore the density dependence of symmetry energy in further detail.