Hi,
To identify if it's the AC adapter, remove the battery then plug in the AC adapter then see i the laptop boots up normally without the battery. If it does that means your battery is faulty but if it doesn't your AC adapter is the culprit. Another test you can do is to get a voltage meter and see if both the battery and AC adapter are giving out the recommended voltage that your laptop needs and you can find those information from the stickers on the back of the laptop and the AC adapter.
Yes, you can still use the laptop as normal even though it's powered just with it's AC adapter.