Error Message: HTTP-Error 500 "Unable to connect to Database" | No Format |
|---|
2021-04-08T09:17:37,383 ERROR [topology-scan_Worker-1]: Error executing call to [/rest/api/uom/ManagedSystem/<system_id>/<object_type>], got status [500 INTERNAL_SERVER_ERROR] with message [
<entry xmlns="http://www.w3.org/2005/Atom" xmlns:ns2="http://a9.com/-/spec/opensearch/1.1/" xmlns:ns3="http://www.w3.org/1999/xhtml">
<id>5fc9dc1c-03be-4e80-b8fb-4f9ea9dc0b4a</id>
<title>HttpErrorResponse</title>
<published>2021-04-08T09:17:46.640+02:00</published>
<author>
<name>IBM Power Systems Management Console</name>
</author>
<content type="application/vnd.ibm.powervm.web+xml; type=HttpErrorResponse">
<HttpErrorResponse:HttpErrorResponse xmlns:HttpErrorResponse="http://www.ibm.com/xmlns/systems/power/firmware/web/mc/2012_10/" xmlns="http://www.ibm.com/xmlns/systems/power/firmware/web/mc/2012_10/" xmlns:ns2="http://www.w3.org/XML/1998/namespace/k2" schemaVersion="V1_0">
<Metadata>
<Atom/>
</Metadata>
<HTTPStatus kb="ROR" kxe="false">500</HTTPStatus>
<RequestURI kxe="false" kb="ROR">/rest/api/uom/ManagedSystem/3870ebf9-dd78-31e5-a91e-719c4f86178b/NetworkBridge</RequestURI>
<ReasonCode kb="ROR" kxe="false">Unknown internal error.</ReasonCode>
<Message kb="ROO" kxe="false">com.ibm.pmc.rest.provider.exceptions.RESTProviderException: Exception While getting SEA:: Unable to connect to Database. </Message>
<RequestBody kb="ROO" kxe="false"/>
<RequestHeaders kxe="false" kb="ROO">
{x-forwarded-server=hmc101.localdomain, x-forwarded-host=172.16.146.192:12443, X-Transaction-ID=XT10011179, host=172.16.146.192:12443, connection=Keep-Alive, x-api-session=9PLm73Uh1Zg84V-wivc1TrsRpNLdrV-VbWeDWSfBEheHxNiuwGEgl_0OWwcLpFlVDaydA2z37DWTvHLMo9McsEwN9X_at8GfE5_ayfeF9Qjzf2EGlzYxJW0G4BzEaznehGmjRR7GfsY3ktUmcte6LT_-JHq5tCqUzfx4nVz6E9wu4dbG2Bo-DhJnkH81b1HLqpjCZyPQlAHf1fqpFOFWr29_7xq-GN_J5tiE1zwSXvY=, x-forwarded-for=172.16.168.84, accept-encoding=gzip,deflate, accept=application/vnd.ibm.powervm.uom+xml;, user-agent=Apache-HttpClient/4.5.12 (Java/11)}
</RequestHeaders>
</HttpErrorResponse:HttpErrorResponse>
</content>
</entry>
] (PowerVmClient) |
BVQ Version: 2021.H1.3 and above Suggested Action: The Postgres DB on VIO servers which the HMC queries to get various system information (like virtual networks, virtual storage, etc.) is broken or Postgres service (vio daemon) is not running. This causes HTTP-500 error messages on both, the HMC and BVQ scanner. The issue can be fixed using the following commands on the VIOS: | No Format |
|---|
ssh padmin@${vios}
$ oem_setup_env
# stopsrc -s vio_daemon
# /usr/sbin/slibclean
# rm -rf /home/ios/CM
# startsrc -s vio_daemon -a '-d 4' |
Error Message: HTTP-Error 500 "Exception while getting SEA" | No Format |
|---|
2021-05-18T19:33:27,290 ERROR [topology-scan_Worker-6]: Error executing call to [/rest/api/uom/ManagedSystem/b8f44367-98bd-377d-8227-7db6208f1c4c/NetworkBridge], got status [500 INTERNAL_SERVER_ERROR] with message [
<entry xmlns="http://www.w3.org/2005/Atom" xmlns:ns2="http://a9.com/-/spec/opensearch/1.1/" xmlns:ns3="http://www.w3.org/1999/xhtml">
<id>7af9ea16-352a-4ab1-890f-1e24405102e7</id>
<title>HttpErrorResponse</title>
<published>2021-05-18T19:33:27.285+02:00</published>
<author>
<name>IBM Power Systems Management Console</name>
</author>
<content type="application/vnd.ibm.powervm.web+xml; type=HttpErrorResponse">
<HttpErrorResponse:HttpErrorResponse xmlns:HttpErrorResponse="http://www.ibm.com/xmlns/systems/power/firmware/web/mc/2012_10/" xmlns="http://www.ibm.com/xmlns/systems/power/firmware/web/mc/2012_10/" xmlns:ns2="http://www.w3.org/XML/1998/namespace/k2" schemaVersion="V1_0">
<Metadata>
<Atom/>
</Metadata>
<HTTPStatus kxe="false" kb="ROR">500</HTTPStatus>
<RequestURI kb="ROR" kxe="false">/rest/api/uom/ManagedSystem/b8f44367-98bd-377d-8227-7db6208f1c4c/NetworkBridge</RequestURI>
<ReasonCode kb="ROR" kxe="false">Unknown internal error.</ReasonCode>
<Message kb="ROO" kxe="false">com.ibm.pmc.rest.provider.exceptions.RESTProviderException: Exception While getting SEA:: The system is currently too busy to complete the specified request. Please retry the operation at a later time. If the operation continues to fail, check the error log to see if the filesystem is full. </Message>
<RequestBody kb="ROO" kxe="false"/>
<RequestHeaders kxe="false" kb="ROO">{x-forwarded-server=hmc3.labwi.sva.de, x-forwarded-host=hmc3.labwi.sva.de:12443, X-Transaction-ID=XT11438532, host=hmc3.labwi.sva.de:12443, connection=Keep-Alive, x-api-session=xnqKwOuPE9APo0rubodXReDeoXN2SnlAXIeEzEu4guge1pd6sg4oCF0WlAE94qpB7NjiX5q8L5xHLJMUuS4LWqRvxopaTucnrOqa6TACGCWhAMYJ4DekkrJtxlpM_s0GkNkoerZ5JSvutYojiYro9N2TNortma44FydeyORKQF260PAUjI2SLytd10mS8PJTpb9uzkbo6h0P0quXOSqRXg==, x-forwarded-for=10.10.120.73, accept-encoding=gzip,deflate, accept=application/vnd.ibm.powervm.uom+xml;, user-agent=Apache-HttpClient/4.5.12 (Java/11)}</RequestHeaders>
</HttpErrorResponse:HttpErrorResponse>
</content>
</entry>
] (PowerVmClient)
|
BVQ Version: 2021.H1.3 and above Suggested Action: Different problems can result in this error. One reason might be a full filesystem (as the error message itself suggests). Another reason might be a dodgy SEA adapter. Please run command | No Format |
|---|
entstat -all entX |
to see if there are Limbo Packets which indicate that the SEA has detected its physical network is not operational.
Error Message: "ObjectNotValidException" | No Format |
|---|
2021-07-19T15:43:45,609 ERROR [persistExecutor_3]: Error during command execution: Error during powervm topology persist execution! [TopoPersist] (AbstractCommandExecutor)
de.sva.bvq.data.grid.api.exception.ObjectNotValidException: Create operation not valid for object of type [pvm_physical_volume_to_virtual_io_server]! pvm_physical_volume_location_code: [[[pvm_physical_volume_location_code] must not be null!][[pvm_physical_volume_location_code] is primary key (not auto generated) and must be set!]] |
BVQ Version: 2021.H1.3 and above Suggested Action: Information provided by AIX and sent via REST do not match (In this case it was an hdisk which looked fine from AIX point of view but reported no location code when queried via REST API). Root cause is not understood but rebooting the VIOS servers - one after the other - probably fixes the issue. |