DBASolved

Subscribe to  DBASolved feed  DBASolved
Helping Oracle DBAs solve problems
Updated: 6 hours 47 min ago

#GoldenGate Procedural Replication Support

Thu, 2017-10-26 17:32

With the release of Oracle GoldenGate 12c (12.3.0.1.x), came the support for Procedural Replication. This option is an enhancement to the integrated replicat which allows you to replicat Oracle-supplied PL/SQL procedures that cannot be replicated as DML or DDL.

Note: In order to replicate these packages and procedures, you need to be on Oracle Database Release 2 (12.2) an later and using the integrated extract and replicat.

To find out what packages and procedures are support, Oracle has provided new data dictionary views that can be used to see what packages and procedures can be used with Oracle GoldenGate. These views are DBA_GG_SUPPORTED_PACKAGES and DBA_GG_SUPPORTED_PROCEDURES. These views can be found in Oracle Database Release 2 (12.2).

As of Oracle GoldenGate 12.3, there are 309 procedures and 36 packages that are support. This number will increase over time.

The packages that are currently supported are:

DBMS_AQ
DBMS_AQADM
DBMS_AQELM
DBMS_AQJMS
DBMS_DBFS_CONTENT_ADMIN
DBMS_DBFS_SFS
DBMS_DBFS_SFS_ADMIN
DBMS_DDL
DBMS_FGA
DBMS_GOLDENGATE_ADM
DBMS_GOLDENGATE_IMP
DBMS_REDACT
DBMS_REDEFINITION
DBMS_RESCONFIG
DBMS_RLS
DBMS_RULE_ADM
DBMS_SQL_TRANSLATOR
DBMS_XDB
DBMS_XDBRESOURCE
DBMS_XDBZ
DBMS_XDB_ADMIN
DBMS_XDB_CONFIG
DBMS_XDB_REPOS
DBMS_XDB_VERSION
DBMS_XMLINDEX
DBMS_XMLSCHEMA
DBMS_XMLSCHEMA_LSB
SDO_META
SDO_META_USER
XS_ACL
XS_ADMIN_UTIL
XS_DATA_SECURITY
XS_DATA_SECURITY_UTIL
XS_NAMESPACE
XS_PRINCIPAL
XS_SECURITY_CLASS

Packages with associated procedures that are currentlly supported are:

DBMS_AQ AQ$_BACKGROUND_OPER
DBMS_AQ AQ$_BACKGROUND_OPER_PAS
DBMS_AQ AQ$_DELETE_DIOT_TAB
DBMS_AQ AQ$_DELETE_HIST_TAB
DBMS_AQ AQ$_DELETE_TIOT_TAB
DBMS_AQ AQ$_DEQUEUE
DBMS_AQ AQ$_ENQUEUE
DBMS_AQ AQ$_INSERT_DIOT_TAB
DBMS_AQ AQ$_INSERT_HIST_TAB
DBMS_AQ AQ$_INSERT_TIOT_TAB
DBMS_AQ AQ$_UPDATE_HIST_TAB
DBMS_AQ AQ$_UPDATE_HIST_TAB_EX
DBMS_AQ BIND_AGENT
DBMS_AQ DEQUEUE
DBMS_AQ DEQUEUE_INTERNAL
DBMS_AQ DEQUEUE_INTERNAL_PAS
DBMS_AQ ENQUEUE
DBMS_AQ ENQUEUE_INT_SHARD
DBMS_AQ ENQUEUE_INT_SHARD_JMS
DBMS_AQ ENQUEUE_INT_UNSHARDED
DBMS_AQ ENQUEUE_INT_UNSHARDED_PAS
DBMS_AQ LISTEN
DBMS_AQ POST
DBMS_AQ REGISTER
DBMS_AQ REGISTRATION_REPLICATION
DBMS_AQ UNBIND_AGENT
DBMS_AQ UNREGISTER
DBMS_AQADM ADD_ALIAS_TO_LDAP
DBMS_AQADM ADD_CONNECTION_TO_LDAP
DBMS_AQADM ADD_SUBSCRIBER
DBMS_AQADM ALTER_AQ_AGENT
DBMS_AQADM ALTER_PROPAGATION_SCHEDULE
DBMS_AQADM ALTER_QUEUE
DBMS_AQADM ALTER_QUEUE_TABLE
DBMS_AQADM ALTER_SHARDED_QUEUE
DBMS_AQADM ALTER_SUBSCRIBER
DBMS_AQADM CREATE_AQ_AGENT
DBMS_AQADM CREATE_EXCEPTION_QUEUE
DBMS_AQADM CREATE_NP_QUEUE
DBMS_AQADM CREATE_QUEUE
DBMS_AQADM CREATE_QUEUE_TABLE
DBMS_AQADM CREATE_SHARDED_QUEUE
DBMS_AQADM DEL_ALIAS_FROM_LDAP
DBMS_AQADM DEL_CONNECTION_FROM_LDAP
DBMS_AQADM DISABLE_DB_ACCESS
DBMS_AQADM DISABLE_PROPAGATION_SCHEDULE
DBMS_AQADM DROP_AQ_AGENT
DBMS_AQADM DROP_QUEUE
DBMS_AQADM DROP_QUEUE_TABLE
DBMS_AQADM DROP_SHARDED_QUEUE
DBMS_AQADM ENABLE_DB_ACCESS
DBMS_AQADM ENABLE_JMS_TYPES
DBMS_AQADM ENABLE_PROPAGATION_SCHEDULE
DBMS_AQADM GET_PROP_SEQNO
DBMS_AQADM GET_REPLAY_INFO
DBMS_AQADM GET_TYPE_INFO
DBMS_AQADM GET_WATERMARK
DBMS_AQADM GRANT_QUEUE_PRIVILEGE
DBMS_AQADM GRANT_SYSTEM_PRIVILEGE
DBMS_AQADM GRANT_TYPE_ACCESS
DBMS_AQADM MIGRATE_QUEUE_TABLE
DBMS_AQADM NONREPUDIATE_RECEIVER
DBMS_AQADM NONREPUDIATE_SENDER
DBMS_AQADM PURGE_QUEUE_TABLE
DBMS_AQADM RECOVER_PROPAGATION
DBMS_AQADM REMOVE_SUBSCRIBER
DBMS_AQADM RESET_REPLAY_INFO
DBMS_AQADM REVOKE_QUEUE_PRIVILEGE
DBMS_AQADM REVOKE_SYSTEM_PRIVILEGE
DBMS_AQADM SCHEDULE_PROPAGATION
DBMS_AQADM SET_WATERMARK
DBMS_AQADM START_QUEUE
DBMS_AQADM START_TIME_MANAGER
DBMS_AQADM STOP_QUEUE
DBMS_AQADM STOP_TIME_MANAGER
DBMS_AQADM UNSCHEDULE_PROPAGATION
DBMS_AQADM VERIFY_QUEUE_TYPES
DBMS_AQADM VERIFY_QUEUE_TYPES_GET_NRP
DBMS_AQADM VERIFY_QUEUE_TYPES_NO_QUEUE
DBMS_AQELM GET_MAILHOST
DBMS_AQELM GET_MAILPORT
DBMS_AQELM GET_PROXY
DBMS_AQELM GET_SENDFROM
DBMS_AQELM GET_TXTIMEOUT
DBMS_AQELM HTTP_SEND
DBMS_AQELM SEND_EMAIL
DBMS_AQELM SET_MAILHOST
DBMS_AQELM SET_MAILPORT
DBMS_AQELM SET_PROXY
DBMS_AQELM SET_SENDFROM
DBMS_AQJMS AQ$_GET_PROP_STAT
DBMS_AQJMS AQ$_GET_TRANS_TYPE
DBMS_AQJMS AQ$_REGISTER
DBMS_AQJMS AQ$_UNREGISTER
DBMS_AQJMS AQ$_UPDATE_PROP_STAT_QNAME
DBMS_AQJMS CLEAR_DBSESSION_GUID
DBMS_AQJMS CLEAR_GLOBAL_AQCLNTDB_CTX_CLNT
DBMS_AQJMS CLEAR_GLOBAL_AQCLNTDB_CTX_DB
DBMS_AQJMS GET_DB_USERNAME_FOR_AGENT
DBMS_AQJMS SET_DBSESSION_GUID
DBMS_AQJMS SET_GLOBAL_AQCLNTDB_CTX
DBMS_AQJMS SUBSCRIBER_EXISTS
DBMS_DBFS_CONTENT_ADMIN EXIM_MOUNT
DBMS_DBFS_CONTENT_ADMIN EXIM_MOUNTP
DBMS_DBFS_CONTENT_ADMIN EXIM_STORE
DBMS_DBFS_CONTENT_ADMIN MOUNTSTORE_LOG
DBMS_DBFS_CONTENT_ADMIN REGISTERSTORE_LOG
DBMS_DBFS_CONTENT_ADMIN UNMOUNTSTORE_LOG
DBMS_DBFS_CONTENT_ADMIN UNREGISTERSTORE_LOG
DBMS_DBFS_SFS NORMALIZEFS
DBMS_DBFS_SFS REORGANIZEFS
DBMS_DBFS_SFS SHRINKFS
DBMS_DBFS_SFS_ADMIN CREATEFILESYSTEM_LOG
DBMS_DBFS_SFS_ADMIN DELETE_ORPHANS_LOG
DBMS_DBFS_SFS_ADMIN DROPFILESYSTEM_LOG
DBMS_DBFS_SFS_ADMIN EXIM_ATTRV
DBMS_DBFS_SFS_ADMIN EXIM_FS
DBMS_DBFS_SFS_ADMIN EXIM_GRANTS
DBMS_DBFS_SFS_ADMIN EXIM_SEQ
DBMS_DBFS_SFS_ADMIN EXIM_SNAP
DBMS_DBFS_SFS_ADMIN EXIM_TABP
DBMS_DBFS_SFS_ADMIN EXIM_TAB_LOG
DBMS_DBFS_SFS_ADMIN EXIM_VOL
DBMS_DBFS_SFS_ADMIN INITFILESYSTEM_LOG
DBMS_DBFS_SFS_ADMIN PARTITION_SEQUENCE_LOG
DBMS_DBFS_SFS_ADMIN RECACHE_SEQUENCE_LOG
DBMS_DBFS_SFS_ADMIN REGISTERFILESYSTEM_LOG
DBMS_DBFS_SFS_ADMIN SETFSPROPERTIES_LOG
DBMS_DBFS_SFS_ADMIN UNREGISTERFILESYSTEM_LOG
DBMS_DDL SET_TRIGGER_FIRING_PROPERTY
DBMS_FGA ADD_POLICY
DBMS_FGA DISABLE_POLICY
DBMS_FGA DROP_POLICY
DBMS_FGA ENABLE_POLICY
DBMS_GOLDENGATE_IMP ACDR_COLUMN
DBMS_GOLDENGATE_IMP ACDR_COLUMN_GROUP
DBMS_GOLDENGATE_IMP ACDR_END
DBMS_GOLDENGATE_IMP ACDR_START
DBMS_GOLDENGATE_IMP ACDR_TABLE
DBMS_REDACT ADD_POLICY
DBMS_REDACT ALTER_POLICY
DBMS_REDACT APPLY_POLICY_EXPR_TO_COL
DBMS_REDACT CREATE_POLICY_EXPRESSION
DBMS_REDACT DISABLE_POLICY
DBMS_REDACT DROP_POLICY
DBMS_REDACT DROP_POLICY_EXPRESSION
DBMS_REDACT ENABLE_POLICY
DBMS_REDACT FPM_MASK
DBMS_REDACT FPM_UNMASK
DBMS_REDACT UPDATE_FULL_REDACTION_VALUES
DBMS_REDACT UPDATE_POLICY_EXPRESSION
DBMS_REDEFINITION ABORT_REDEF_TABLE
DBMS_REDEFINITION ABORT_ROLLBACK
DBMS_REDEFINITION ABORT_UPDATE
DBMS_REDEFINITION CAN_REDEF_TABLE
DBMS_REDEFINITION COPY_TABLE_DEPENDENTS
DBMS_REDEFINITION EXECUTE_UPDATE
DBMS_REDEFINITION FINISH_REDEF_TABLE
DBMS_REDEFINITION REDEF_TABLE
DBMS_REDEFINITION REGISTER_DEPENDENT_OBJECT
DBMS_REDEFINITION ROLLBACK
DBMS_REDEFINITION SET_PARAM
DBMS_REDEFINITION START_REDEF_TABLE
DBMS_REDEFINITION SYNC_INTERIM_TABLE
DBMS_REDEFINITION UNREGISTER_DEPENDENT_OBJECT
DBMS_RESCONFIG ADDREPOSITORYRESCONFIG
DBMS_RESCONFIG ADDRESCONFIG
DBMS_RESCONFIG APPENDRESCONFIG
DBMS_RESCONFIG DELETEREPOSITORYRESCONFIG
DBMS_RESCONFIG DELETERESCONFIG
DBMS_RULE_ADM GRANT_OBJECT_PRIVILEGE
DBMS_RULE_ADM GRANT_SYSTEM_PRIVILEGE
DBMS_RULE_ADM REVOKE_OBJECT_PRIVILEGE
DBMS_RULE_ADM REVOKE_SYSTEM_PRIVILEGE
DBMS_SQL_TRANSLATOR CLEAR_SQL_TRANSLATION_ERROR
DBMS_SQL_TRANSLATOR CREATE_PROFILE
DBMS_SQL_TRANSLATOR DEREGISTER_ERROR_TRANSLATION
DBMS_SQL_TRANSLATOR DEREGISTER_SQL_TRANSLATION
DBMS_SQL_TRANSLATOR DROP_PROFILE
DBMS_SQL_TRANSLATOR ENABLE_ERROR_TRANSLATION
DBMS_SQL_TRANSLATOR ENABLE_SQL_TRANSLATION
DBMS_SQL_TRANSLATOR REGISTER_ERROR_TRANSLATION
DBMS_SQL_TRANSLATOR REGISTER_SQL_TRANSLATION
DBMS_SQL_TRANSLATOR SET_ATTRIBUTE
DBMS_SQL_TRANSLATOR SET_ERROR_TRANSLATION_COMMENT
DBMS_SQL_TRANSLATOR SET_SQL_TRANSLATION_COMMENT
DBMS_SQL_TRANSLATOR SET_SQL_TRANSLATION_MODULE
DBMS_XDBZ ADD_APPLICATION_PRINCIPAL
DBMS_XDBZ CHANGE_APPLICATION_MEMBERSHIP
DBMS_XDBZ DELETE_APPLICATION_PRINCIPAL
DBMS_XDBZ DISABLE_HIERARCHY
DBMS_XDBZ ENABLE_HIERARCHY
DBMS_XDBZ SET_APPLICATION_PRINCIPAL
DBMS_XDB_ADMIN CREATENONCEKEY
DBMS_XDB_ADMIN INSTALLDEFAULTWALLET
DBMS_XDB_ADMIN MOVEXDB_TABLESPACE
DBMS_XDB_ADMIN REBUILDHIERARCHICALINDEX
DBMS_XDB_CONFIG ADDAUTHENTICATIONMAPPING
DBMS_XDB_CONFIG ADDAUTHENTICATIONMETHOD
DBMS_XDB_CONFIG ADDTRUSTMAPPING
DBMS_XDB_CONFIG ADDTRUSTSCHEME
DBMS_XDB_CONFIG CLEARHTTPDIGESTS
DBMS_XDB_CONFIG DELETEAUTHENTICATIONMAPPING
DBMS_XDB_CONFIG DELETEAUTHENTICATIONMETHOD
DBMS_XDB_CONFIG DELETETRUSTMAPPING
DBMS_XDB_CONFIG DELETETRUSTSCHEME
DBMS_XDB_CONFIG ENABLECUSTOMAUTHENTICATION
DBMS_XDB_CONFIG ENABLECUSTOMTRUST
DBMS_XDB_CONFIG ENABLEDIGESTAUTHENTICATION
DBMS_XDB_CONFIG ISGLOBALPORTENABLED
DBMS_XDB_CONFIG SETDYNAMICGROUPSTORE
DBMS_XDB_CONFIG SETGLOBALPORTENABLED
DBMS_XDB_CONFIG SETHTTPCONFIGREALM
DBMS_XDB_VERSION CHECKIN_INT
DBMS_XDB_VERSION CHECKOUT
DBMS_XDB_VERSION MAKEVERSIONED_INT
DBMS_XDB_VERSION UNCHECKOUT_INT
DBMS_XMLINDEX CREATEDATEINDEX
DBMS_XMLINDEX CREATENUMBERINDEX
DBMS_XMLINDEX DROPPARAMETER
DBMS_XMLINDEX MODIFYPARAMETER
DBMS_XMLINDEX PROCESS_PENDING
DBMS_XMLINDEX REGISTERPARAMETER
DBMS_XMLINDEX SYNCINDEX
DBMS_XMLSCHEMA COMPILESCHEMA
DBMS_XMLSCHEMA COPYEVOLVE
DBMS_XMLSCHEMA DELETESCHEMA
DBMS_XMLSCHEMA GENERATEBEAN
DBMS_XMLSCHEMA INPLACEEVOLVE
DBMS_XMLSCHEMA PURGESCHEMA
DBMS_XMLSCHEMA_LSB COPYEVOLVE
DBMS_XMLSCHEMA_LSB REGISTERSCHEMA_BLOB
DBMS_XMLSCHEMA_LSB REGISTERSCHEMA_OID
DBMS_XMLSCHEMA_LSB REGISTERSCHEMA_STR
DBMS_XMLSCHEMA_LSB REGISTERSCHEMA_XML
SDO_META CHANGE_ALL_SDO_GEOM_METADATA
SDO_META CHANGE_IMP_SDO_GEOM_METADATA
SDO_META DELETEGEORSYSENTRY
SDO_META DELETE_ALL_SDO_GEOM_METADATA
SDO_META DELETE_TOPO_METADATA
SDO_META INSERTGEORSYSENTRY
SDO_META INSERT_ALL_SDO_GEOM_METADATA
SDO_META INSERT_TOPO_METADATA
SDO_META UPDATE_TOPO_METADATA
SDO_META_USER CREATEDMLTRIGGER
SDO_META_USER CREATE_TOPO_BTREE
XS_ACL ADD_ACL_PARAMETER
XS_ACL APPEND_ACES
XS_ACL CREATE_ACL
XS_ACL DELETE_ACL
XS_ACL REMOVE_ACES
XS_ACL REMOVE_ACL_PARAMETERS
XS_ACL SET_DESCRIPTION
XS_ACL SET_PARENT_ACL
XS_ACL SET_SECURITY_CLASS
XS_ADMIN_UTIL DROP_SCHEMA_OBJECTS
XS_ADMIN_UTIL GRANT_SYSTEM_PRIVILEGE
XS_ADMIN_UTIL REVOKE_SYSTEM_PRIVILEGE
XS_DATA_SECURITY ADD_COLUMN_CONSTRAINTS
XS_DATA_SECURITY APPEND_REALM_CONSTRAINTS
XS_DATA_SECURITY APPLY_OBJECT_POLICY
XS_DATA_SECURITY CREATE_ACL_PARAMETER
XS_DATA_SECURITY CREATE_POLICY
XS_DATA_SECURITY DELETE_ACL_PARAMETER
XS_DATA_SECURITY DELETE_POLICY
XS_DATA_SECURITY DISABLE_OBJECT_POLICY
XS_DATA_SECURITY ENABLE_OBJECT_POLICY
XS_DATA_SECURITY REMOVE_COLUMN_CONSTRAINTS
XS_DATA_SECURITY REMOVE_OBJECT_POLICY
XS_DATA_SECURITY REMOVE_REALM_CONSTRAINTS
XS_DATA_SECURITY SET_DESCRIPTION
XS_NAMESPACE ADD_ATTRIBUTES
XS_NAMESPACE CREATE_TEMPLATE
XS_NAMESPACE DELETE_TEMPLATE
XS_NAMESPACE REMOVE_ATTRIBUTES
XS_NAMESPACE SET_DESCRIPTION
XS_NAMESPACE SET_HANDLER
XS_PRINCIPAL ADD_PROXY_TO_DBUSER
XS_PRINCIPAL ADD_PROXY_USER
XS_PRINCIPAL CREATE_DYNAMIC_ROLE
XS_PRINCIPAL CREATE_ROLE
XS_PRINCIPAL CREATE_USER
XS_PRINCIPAL DELETE_PRINCIPAL
XS_PRINCIPAL ENABLE_BY_DEFAULT
XS_PRINCIPAL ENABLE_ROLES_BY_DEFAULT
XS_PRINCIPAL GRANT_ROLES
XS_PRINCIPAL REMOVE_PROXY_FROM_DBUSER
XS_PRINCIPAL REMOVE_PROXY_USERS
XS_PRINCIPAL REVOKE_ROLES
XS_PRINCIPAL SET_ACL
XS_PRINCIPAL SET_DESCRIPTION
XS_PRINCIPAL SET_DYNAMIC_ROLE_DURATION
XS_PRINCIPAL SET_DYNAMIC_ROLE_SCOPE
XS_PRINCIPAL SET_EFFECTIVE_DATES
XS_PRINCIPAL SET_GUID
XS_PRINCIPAL SET_PASSWORD
XS_PRINCIPAL SET_PROFILE
XS_PRINCIPAL SET_USER_SCHEMA
XS_PRINCIPAL SET_USER_STATUS
XS_PRINCIPAL SET_VERIFIER
XS_SECURITY_CLASS ADD_IMPLIED_PRIVILEGES
XS_SECURITY_CLASS ADD_PARENTS
XS_SECURITY_CLASS ADD_PRIVILEGES
XS_SECURITY_CLASS CREATE_SECURITY_CLASS
XS_SECURITY_CLASS DELETE_SECURITY_CLASS
XS_SECURITY_CLASS REMOVE_IMPLIED_PRIVILEGES
XS_SECURITY_CLASS REMOVE_PARENTS
XS_SECURITY_CLASS REMOVE_PRIVILEGES
XS_SECURITY_CLASS SET_DESCRIPTION

 

Enjoy!!!


Filed under: Golden Gate
Categories: DBA Blogs

#GoldenGate/#Database Patches

Wed, 2017-10-25 16:35

In searching for patches for Oracle GoldenGate you have to also look for Oracle Database patches. To make this search a bit easier, some of the great guys on the Oracle GoldenGate team has created a My Oracle Support (MOS) note or two. These notes should be upkept over time. For now, just know these are reference points for you to find out what patches are required for your Oracle GoldenGate environments!

Doc Id: 2193391.1 – Latest GoldenGate/Database (OGG/RDBMS) Patch recommendations

Doc Id: 1557031.1 – Oracle GoldenGate — Oracle RDBMS Server Recommended Patches

Enjoy!!!


Filed under: Golden Gate
Categories: DBA Blogs

OGG-00868/ORA-01291 – Missing log files with Integrated Extract

Sun, 2017-10-22 10:05

Oracle GoldenGate is reliant on the Oracle Redo Logs and Archive Logs when capturing transactions. Since I do a lot of testing on VMs (limited space) and in the cloud (limited space … don’t want to burn to much $), I often delete my archive logs. Normally this is not a problem; however, every once-in-awhile I delete more archivelogs than I should. This throws the IE into a state where it will not start because of OGG-00868/ORA-01291 – Missing Log Files.

On some level this is to be expected, but when you are using IE you have to remember that the extract is registered with the database. Since I’m using Integrated Extract, we have to reset how the extract is registered with the database. The below steps will show you how this should be done:

Note: Registering/Unregistering process have to be done at the container database (CDB) level.

adminclient> dblogin useridalias <alias> domain <domain>
adminclient> stop extract <extract_name>
adminclient> unregister extract <extract_name> database
adminclient> register extract <extract_name> database container <container_name>
adminclient> start extract <extract_name>
adminclient> info extract <extract_name>

Once the extract is re-registered with the database; the integrated extract will start.

Enjoy!!!


Filed under: Golden Gate
Categories: DBA Blogs

#GoldenGate Microservices (4 of 5) … Receiver Service

Mon, 2017-09-25 12:23

This is post 4 of a 5 part post related to Oracle GoldenGate 12.3 Microservices. This series of posts will provide some details over the new graphical user interface (GUI) that has been built into Oracle GoldenGate 12.3.

The Receiver Serivice is the least complicated of the microservices with Oracle GoldenGate 12.3 Microservices. This service is taking the place of the collectors in classic architectures. When you fist login to the Reciever Service, you land on the Overview page (Figure 1). On this page, you can clearly see that the host were it is running on what trail file it is writing to.

Figure 1:

If you click on the “Action” button, you willl be presented with a “Details” option. This option takes you to the Path Information page. On this page, you can see the Network Statistics and File IO Statistics (Figure 2). All this information provides you a view into how your transactions are being written to the trail file and at what rate.

Figure 2:

Lastly, if you go back to the overview page. On the context menu, just like other pages, you have a Diagnosis & Reports page (Figure 3). This page, provides you a quick and easy look at the log file associated with the services. It is color coded (greed, yellow, red) to help you quickly identify what needs to be looked at.

Figure 3:

Enjoy!!!


Filed under: Golden Gate
Categories: DBA Blogs

#GoldenGate Microservices (3 of 5) … Distribution Service

Mon, 2017-09-25 09:44

This is post 3 of a 5 part post related to Oracle GoldenGate 12.3 Microservices. This series of posts will provide some details over the new graphical user interface (GUI) that has been built into Oracle GoldenGate 12.3.

The third part of setting up the replication enviornment with Microservices is to understand what the Distributuion Server is. The component is the replacement for the Data Pump Extract in classic architecture. Distribution Server does all the same things as the Data Pump Extract except with from transformations. If you use transformations in the Data Pump, you really should relook where this is being done.

Note: Transformations should be done at the source, target, or with Oracle Data Integrator.

Distribution Server is your path way for shipping trail files from source to target in the Microservices architecture. Let’s take a closer look at this service.

Upon accessing the Distribution Server page, you see a blank page (Figure 1). This is the overview page and where you will configure you distribution paths needed for replication.

Figure 1:

To begin adding a distribution path, click the plus ( + ) sign on the right hand side of the page. This will take you to the Add Path page (Figure 2). On the Add Path page, you will provide the details for the path. These details include extract/trail file to read from, protocol to use for transmissions, log positioning and many other options. A lot of the items you could do in the data pump extract, can be configured on this page as well.

Figure 2:

After adding a distribution path, you will be able to see clearly where the transactions are coming from and being shipped to (Figure 3). You will also see what database, extract, and trail if being used for the transactions.

Figure 3:

If you click on the “Action” button, you are taken to a menu where you can perform operations on the Distribution Path. There is an option for “Details”. By looking at the details, you can quickly see, in more detail, what is happening with the Distrubtuion Path. You can see the source and target, database, extract, trail name, source trail file, as well as many other item (Figure 4). Additonally, on the details page, you can see the statistics for what is running through the Distribution path.

Figure 4:

Lastly, if you return to the context menu on the left hand side, you will see an option for “Diagnosis & Reports”. Here is where you will be able to take a look at the report file for the Distribution Server (Figure 5). This page provides you with a date sorted view of the log file for easy reading. You will also be able to quickly find items that are wrong or warnings by the color indicator used with the log file.

Figure 5:

Enjoy!!!


Filed under: Golden Gate
Categories: DBA Blogs

#GoldenGate sessions at #OOW17

Thu, 2017-09-21 14:52

We are a little over a week out from Oracle Open World 2017; hopefully you are filling out your schedules! This year there is a lot of items going on; so I figured I would help you out and point out where you can find the Oracle GoldenGate sessions this year! With the release of Oracle GoldenGate 12.3, there is a lot of new features and updates. So check them out (in no particular order)!

Sessions:

  • Oracle GoldenGate Product Update and Strategy
    • Tuesday, Oct 03, 5:45 p.m. – 6:30 p.m. | Moscone West – Room 3003
  • Maximizing Availability for Oracle GoldenGate Microservices
    • Tuesday, Oct 03, 4:45 p.m. – 5:30 p.m. | Moscone West – Room 3014
  • Oracle GoldenGate for Big Data
    • Wednesday, Oct 04, 4:30 p.m. – 5:15 p.m. | Moscone West – Room 3005
  • Deep Dive into Automating Oracle GoldenGate Using the New Microservices
    • Monday, Oct 02, 1:15 p.m. – 2:00 p.m. | Moscone West – Room 3010
  • Accelerate Cloud Onboarding Using Oracle GoldenGate Cloud Service
    • Tuesday, Oct 03, 3:45 p.m. – 4:30 p.m. | Moscone West – Room 3024
  • Oracle Data Integrator and Oracle GoldenGate for Big Data
    • Wednesday, Oct 04, 11:30 a.m. – 12:30 p.m. | Hilton San Francisco Union Square (Ballroom Level) – Continental 4
  • Replicate Data Across Data Centers with Equinix and Oracle GoldenGate
    • Wednesday, Oct 04, 10:30 a.m. – 10:50 a.m. | The Exchange @ Moscone South – Showcase Theater 1
  • An Enterprise Databus: Oracle GoldenGate in the Cloud Working with Kafka and Spark
    • Wednesday, Oct 04, 3:30 p.m. – 4:15 p.m. | Moscone West – Room 3003
  • Oracle GoldenGate Cloud Service: Real-Time Data Replication in the Cloud
    • Monday, Oct 02, 6:00 p.m. – 7:00 p.m. | Hilton San Francisco Union Square (Ballroom Level) – Continental 4
  • Best Practices and Deep Dive on Oracle GoldenGate 12.3 Microservices at Cloud
    • Wednesday, Oct 04, 3:30 p.m. – 4:15 p.m. | Moscone West – Room 3011
  • How to Replicate Data in the Cloud in Real Time
    • Wednesday, Oct 04, 11:00 a.m. – 1:00 p.m. | Hilton San Francisco Union Square (Lobby Level) – Golden Gate 2/3
  • Oracle Data Integration Platform Cloud Strategy and Roadmap
    • Monday, Oct 02, 12:15 p.m. – 1:00 p.m. | Moscone West – Room 3024
  • Oracle Data Integration Platform Empowers Enterprise-Grade Big Data Solutions
    • Monday, Oct 02, 4:45 p.m. – 5:30 p.m. | Marriott Marquis (Golden Gate Level) – Golden Gate C1/C2
  • Kafka’s Role in Implementing Oracle’s Big Data Reference Architecture  
    • Sunday, Oct 01, 1:45 p.m. – 2:30 p.m. | Marriott Marquis (Yerba Buena Level) – Salon 12
  • Oracle Maximum Availability Architecture Best Practices: Oracle Database 12 c
    • Tuesday, Oct 03, 5:45 p.m. – 6:30 p.m. | Moscone West – Room 3006

As you can tell, there is a lot of information that is going to be helpful for your Oracle GoldenGate, BigData and various other areas. Make sure you check all these out.

Enjoy!!


Filed under: Golden Gate
Categories: DBA Blogs

#GoldenGate and #Docker!

Wed, 2017-09-13 22:55

Check this out … Oracle GoldenGate is now avaliable on Docker! All the details you need to get up and running on an image is included in Oracle’s public github.

https://github.com/oracle/docker-images/tree/master/OracleGoldenGate

Enjoy!!!


Filed under: Docker, Golden Gate
Categories: DBA Blogs

#GoldenGate Microservices (2 of 5) … Administration Service

Tue, 2017-09-12 14:30

This is post 2 of a 5 part post related to Oracle GoldenGate 12.3 Microservices. This series of posts will provide some details over the new graphical user interface (GUI) that has been built into Oracle GoldenGate 12.3.

The second interface of the Microservices Architecture that you need to understand is the Administration Service. This service is the interface where you will setup the capture (extract) process and the apply (replicat) process. On the left hand side of the page, in the context menu, you will see a few options … overview, configuration, diagnosis and administrator. These are the primary areas that you interact with the administrator server.

On the overview page, as previously mentioned you can setup your capture (extract) and apply (replicat) processes. You also get a view of the critical events that is are happening within the environment (Figure 1).

Figure 1:

On the configuration page, you will be able to configure credentials that are needed for your replication environment. The addtions of the credential page allows you to create your standard logins for the goldengate user within the database, plus what I’m calling “protocol” users that can be used within the replication environment. Next is the maintenance tab; this tab you can setup tasks like auto start, auto restart, purging trail files, and purge tasks. Additionally, you can add a master key that is used with encryption. Lastly the parameter files tab, will list all the parameter files that is configured in the environment. From here you can edit all the parameters files (Figure 2).

Figure 2:

Next is the diagnosis page. This page is where you can see what is happening in the log file for the Administraton Service. Items on this page will be color coded, green, yellow and red. This give you an indication of what is happening on what you should be looking at (Figure 3).

Figure 3:

Lastly, you have the Administrator page. This page is where you can create users that can access the Oracle GoldenGate environment. When you create a user for access purposes, you get four roles to choose from … Security, Administrator, Operator, and User. Each one of these roles provide a varying level of access (Figure 4).

Note: Do not delete the security user that is created during the intital install of the software.

Figure 4:

Enjoy!!!


Filed under: Golden Gate
Categories: DBA Blogs

#GoldenGate Microservices (1 of 5) … ServiceManager

Mon, 2017-09-11 14:30

This is post 1 of a 5 part post related to Oracle GoldenGate 12.3 Microservices. This series of posts will provide some details over the new graphical user interface (GUI) that has been built into Oracle GoldenGate 12.3.

The first of the Oracle GoldenGate Microservices that you need to understand is that ServiceManager. This service is main interface into the Oracle GoldenGate environment. From the HTML5 page that is provided with ServiceManager, you can quickly get an overview of what is running on the server that Oracle GoldenGate is running on. Additonally the ServiceManager acts as the watch dog process for the environment. This means that if a service was to go down, it can be restarted via the ServiceManager.

The ServiceManager can be configured in three different modes. These modes range from manual to Real Application Cluster (RAC) aware. These modes are:

  1. Manual
  2. Deamon
  3. XAG

These configuration options are selectable during the inital run of Oracel GoldenGate Configuration Assistant (oggca.sh) (Figure 1). If you wanted to use it manually, you woudn’t select anything on this screen. If you want to run it on a single server, than the deamon option is a good choice. For the XAG option, you need to make sure you have the current XAG agent (here).

Figure 1:

After ServiceManage is installed, then you will be able to access the interface via a URL (Figure 2). In the example, you will see that I’m usign port 16000. This is not the default port, but a port that I had configured during the installation using Oracle GoldenGate Configuration Assistant.

Figure 2:

Once the ServerManager page is up, then you just need to login with the administrator account you created during installation of ServiceManager. Keep in mind, if you do not know the userid or password, the consult the people who initally configured the ServiceManager.

After logging in, you are taken directly to the overview page. This page has a few items of importance for the Oracle GoldenGate environment. First you will notice on the left hand side of the page, is a context menu that provides a link to Overview and Diagnostics. This is basically a navigation menu that will be in every microservice (different per service). Then notice at the top of the page, a summary of processes that are running, stopped, or other Lastly, at the middle and bottom of the page, you see the services and deployments that are associated with the installation (Figure 3).

Figure 3:

If you want to know details of the ServiceManager and/or deployments, you can click on the associated deployment under Deployments. This will take you to a summary page, where you can identify items releated to the deployment and change the $OGG_HOME (more on this later).

As you can tell, we have made some great improvments to the Oracle GoldenGate product. Hopefully, this post has you excited to see what else is hiding in the product and my next couple of posts.

Enjoy!!!


Filed under: Golden Gate
Categories: DBA Blogs

Events to round out 2017 … where I’ll be!

Mon, 2017-09-11 08:23

The first half of 2017 started off with me joining Oracle and getting the first looks at the new Microservices Architecture for Oracle GoldenGate. So where does this leave me for the second half of 2017?

As some of you may know, I’m currently working on a Oracle GoldenGate 12.3 roadshow; where I’m explaining the new features in 12.3 and giving attendees hands-on to the new product. After two stops on the roadshow, so far, we have seen so much excitement for this new architecture. It is going to change the way we, as a community, replicate data both on-premise, in the cloud, and in hybrid architecture.

Check out the new Microservice Architecture by downloading it from OTN (here).

As we continue the roadshow, we will be hitting a few more cities in the US before moving on to EMEA in November. Once we get to EMEA, there will be some cool places we plan on holding events and/or meeting customers. The excitement for Oracle GoldenGate 12.3 continues to grow as we attend more cities.

If you want to know the cities in the US, please see this blog post (here).

On top of the US and EMEA legs of the Oracle GoldenGate 12.3 roadshow, I will be attending two conferences in EMEA. I’m sure you can guess at which two … DOAG and UKOUG. This will my first time at both events and I hope I get to see a lot of great friends and make many new ones.

At DOAG, I will be speaking, on November 22, 2017 @ 08:00, about Oracle GoldenGate Peformance Tuning. Additionally, I will have another product manager in tow (hopefully) so attendees will have direct access to the PMs who are responsible for the core Oracle GoldenGate product.

At UKOUG, I will be speaking, need to confirm date, about the Conflict, Detection and Resolution (CDR) in Oracle GoldenGate 12.3 Both the manual process and the new Automatic CDR. Then if needed, by the UKOUG board, I may have another Oracle GoldenGate 12.3 presentation.

All in all, 2017 is rounding out to be a great first year at Oracle along with many first on top of that. Looking forward to seeing everyone on the roadshow!

Enjoy!!!


Filed under: Golden Gate
Categories: DBA Blogs

#GoldenGate Classic vs Microservices Architecture

Fri, 2017-08-25 23:30

With the release of Oracle GoldenGate 12c (12.3.0.1.0) came the introduction of a new architecture that can be used to replicat your business information. This architecture had many different names over the years as it was being developed; which we (Oracle) finally settled on the name of “Microservices” Architecture as we got closer to general release. There are many benefits to the Microservices Architecture and these benefits should give you pause to look closely at this new feature for Oracle GoldenGate 12c.

Before we get into the Microservices Architecture, let’s review the Classic Architecture. In the below image, you see a pretty standard Oracle GoldenGate implementation.

In this architecture, the primary access into the Oracle GoldenGate enviornmentis is through the GoldenGate Service Command Interface (GGSCI). After logging into GGSCI, you can interact and administrate the associated processes, i.e. Manager, Extract (Capture), Data Pump, and Replicat (Apply). The Collectors are pretty much hidden on the target systems, but they are there; just not seen through GGSCI. In this architecture, you data is replicated over TCP/IP between the Data Pump process and Collectors using local and remote trail files.

The downside to this architecture is that, in order to administer the environment, you have to physically login to the server where Oracle GoldenGate is running. With this requirement, many organizations were restrictive on who had access to the server and often caused a debate over who were truely the owners of Oracle GoldenGate.

Althought the Classic Architecture of Oracle GoldenGate has been a bedrock of replication for nearly 20 years, we (Oracle) wanted to leverage that bedrock to transform the way we (industries) replicate data today. This lead to the more flexible and super scalable Microservices Architecture. As you may have guessed, “Microservices” is the mechanism that we are suing to provide access to the “RESTful API” end points. By using RESTful APIs, we (Oracle) have taken a huge leap forward in the replication space. We have broken down the limitations we had around administration and access, while at the same time remaining true to the bedrock that is the foundation of Oracle GoldenGate.

The below image is a view of a simple Oracle GoldenGate Microservices Architecture, for you to review.

As you will notice there are some traditional components of Oracle GoldenGate missing, while there are still extracts (capture), trail files, and replicats (apply). This is due to being a completely new replication architecture which had many benefits. A few of these benefits are:

  • Remote Administration
  • SSL Support
  • HTML 5 webpages for each service/server
  • Additional replication protocols (WSS, WS, UDT, OGG)
  • Real-time Performance Metrics

In order to understand the Microservices Architecture, you have to understand what each of the servers (or services) provide within the architecture. So, let’s take a moment and talk about these items starting with the ServiceManager.

ServiceManager:
The ServiceManager is the watchdog process for the architecture on each server in the replication enviornment. Ideally, you should only have one of these processes running. This process can be configured to run in one of 3 ways. These ways are:

  • Manually
  • As a daemon
  • Integrated with XAG

While the ServiceManager is running, this process will be the main entry point into the Oracle GoldenGate environment. During the configuration process, you will be asked to assign ports for each the servers to run on. The ServiceManager will be the first port you assign. From the HTML5 page of the ServiceManager, you will be able to see all of your deployment homes and associated servers.

AdminServer:
The AdminServer is the service that will take the place GGSCI (don’t worry, we still have a command line in this architecture) and Manager in the Classic Architecture. From here, you will be able to setup your credential store, extract and replicats. Most Oracle GoldenGate Administrators will spend their time here. Additionally, from this service you can drill into the running process and review current status, statistics, parameter files, and report file. Making your administration in general much simpler.

DistributionServer:
The DistributionServer is the replacement for the Data Pump Extract. The service performs all the same functionality as the Data Pump Extract with the exception of transformations. Besides providing all the same functionality, you also get a visual representation of where your trail file is being read from and shipped to. It is very clear to see from the overview page of this services. As you dig into the details of the DistributionServer, you can see the statistics on what is being read and written to trail files and adjust TCP/IP items within the distribution path.

ReceiverServer:
The ReceiverServer is the replacement for the Collectors. The whole job of the ReceiverServer is to accept transmissions from the DistributionServer and write out the remote trail files. From the overview page of this service, you can clearly see where the information is coming from and what trail it is writing to. Just like the DistributionServer, if you look at the details of this service you can see alot of useful information.

Lastly, is the most interesting of the services with the Microservices Architecture. This would be:

Performance Metrics Server:
Finally, we (Oracle) have provide a real-time performance monitoring services with Oracle GoldenGate. Before you get all happy about having a new way to monitor performance, you must have a license for the Oracle Managment Pack for GoldenGate before you can use the GUI or associated metric APIs. If you have that in place, there is so much performance metric information you can retrieve and use in both the GUI and APIs. I would encourage you to take a look.

With that my friends, hopefully, you are a bit excited about using the new Microservices Architecture. There is so much you can do with this architecture and it is going to change how we replicat data, both on-primese, in the cloud, and in hybrid environments.

Enjoy!!!


Filed under: Golden Gate
Categories: DBA Blogs

5 City #GoldenGate Tour

Fri, 2017-08-18 19:59

Oracle GoldenGate is the best replication platform on the market, so much so, the product management team is hitting the road to show customers!  Over the next two months (August 2017 /September 2017) we will be taking to the road to provide customers with hands-on experience of the latest release.  These events will be limited in space and many have filled up already; but if you have the opportunity to attend these will be great events!

The cites and dates that we will be attending (US Leg):

August 23, 2017 – San Francisco, Oracle Headquarters, Redwood Shores
August 29, 2017 – Atlanta, Ga, Oracle Offices
September 12, 2017 – Chicago, Il, Willis Tower (Oracle Offices)
September 13, 2017 – Chicago, Il, Deerfield (Oracle Office)
September 19, 2017 – Dallas, TX
September 26, 2017 – Boston, MA 

If you are interested in attending one of these events, please contact Patrick Cassidy (patrick.cassidy@oracle.com) to register.  Hurry seats are filling up, if not already filled up!!!

Look forward to seeing you at one of the shows.

Note: for the international people reading, we have not forgotten about you.  We are planning on seeing you soon!

Enjoy!!


Filed under: Golden Gate
Categories: DBA Blogs

Oracle #GoldenGate 12.3.0.1 has arrived!

Fri, 2017-08-18 18:46

As of today, August 18, 2017, the latest release of Oracle GoldenGate 12c (12.3.0.1.0) is avaliable for download! You should checkout the new Microservices Architecture!

You can find the links to download Oracle GoldenGate 12c (12.3.0.1.0) at this link: http://www.oracle.com/technetwork/middleware/goldengate/downloads/index.html

This release bring many new features to Oracle Goldengate:

GoldenGate 12.3 Platform Features – All Platforms For the Oracle Database

Support for Oracle Database 12.2.0.1
Oracle Database 12.2.0.1 provides many exciting features for organizations to use. Oracle GoldenGate 12.3 is designed to leverage many of these features in Oracle Database 12.2.0.1 as well. Organizations will have a fully supported and integrated replication framework that provides organizations with performance and throughput enhancements within the Integrated Capture, Integrated Apply and many others processes.

-Microservices Architecture

A new services based architecture that simplifies configuration, administration and monitoring for large scale and cloud deployments. The RESTFul services enable secure remote access using role based authorization over HTTPS and Websocket (streaming) protocols. Each service also has an embedded HTML5 browser based UI for better user experience in addition to traditional command line access for ggsci style scripting/automation. It enables Applications to embed, automate, and orchestrate GoldenGate across the enterprise.

Parallel Replicat
Highly scalable apply engine for the Oracle database that can automatically parallelize the apply workload taking into account dependencies between transactions. Parallel Replicat provides all the benefits of Integrated Replicat performing the dependency computation and parallelism outside the database. It parallelizes the reading and mapping of trail files and provides the ability to apply large transactions quickly in Oracle Database 11g (11.2.0.4) and later.

Automatic Conflict-Detection-Resolution (CDR) without application changes
Quickly enable active/active replication while ensuring consistent conflict-detection-resolution (CDR) without modifying application or database structure. With automatic CDR you can now configure and manage Oracle GoldenGate to automate conflict detection and resolution when it is configured in Oracle Database 12c Release 2 (12.2) and later.

Procedural Replication to enable simpler application migrations and upgrades
Procedural Replication in Oracle GoldenGate allows you to replicate Oracle-supplied PL/SQL procedures, avoiding the shipping and applying of high volume records usually generated by these operations.

Database Sharding
Oracle Sharding is a scalability and availability feature designed OLTP applications that enables distribution and replication of data across a pool of Oracle databases that share no hardware or software. The pool of databases is presented to the application as a single logical database. Data redundancy is managed by Oracle GoldenGate via Active-Active replication that is automatically configured and orchestrated through the database engine invoking the RestFul API’s.

Fast Start Capture
Fast Start Capture is a new feature for Integrated Capture that will improve overall performance and enable you to quickly start capturing and replicating transactions.

For SQL Server

Introducing a new, CDC based Capture

Oracle GoldenGate 12.3 will introduce a new Change Data Capture based Extract, which offers new functional advantages over our existing transaction log based capture method.Benefits include:

  •   Capture from SQL Server 2016
  •   Remote Capture
  •   Transparent Data Encryption (TDE) support

Certification to capture, from an AlwaysOn primary and/or synchronous secondary database
With an increase in uptake of our customers running their application critical databases in an AlwaysOn environment, Oracle GoldenGate 12.3 is the first version to certify capture from either the Primary database, or a read-only Synchronous Secondary database.

-Delivery to SQL Server 2016 Enterprise Edition

For DB2 z/OS

Remote Execution
The new remote execution includes both remote capture and delivery for DB2 z/OS. Running Oracle GoldenGate off the z/OS server significantly reduces the MIPS consumption and allows the support of AES encryption and credential store management.

For DB2 i

Support for IBM i 7.3
Oracle GoldenGate supports the latest DB2 for i platform.

For MySQL

– DDL replication between MySQL Databases
With the DDL replication between MySQL databases, there is no need to stop Oracle GoldenGate replication when there are DDL changes on the source database.

Enjoy!!


Filed under: Golden Gate
Categories: DBA Blogs

#GoldenGate Manager Access Rules

Sat, 2017-06-10 22:21

Starting in Oracle GoldenGate 12.2.0.1, the manager process is more secure than in previous releases of Oracle GoldenGate. After the inital install and running of the Manager process, you may run into the following warning message upon a restart of the Oracle GoldenGate environment:

WARNING OGG-01877 Missing explicit accessrule for server collector.

The inital description of the error message doesn’t make much sense. Using OGGERR, you can find out more about this error message:

$ oggerr ogg-01877
01877, 00000, "Missing explicit accessrule for server collector."
// *Cause: There was no explict ACCESSRULE specified for SERVER.
// *Action: Only allow accessing SERVER from hosts where data pump(s) might

// send trail files to this host.

As you can tell, the error message is pointing you to the ACCESSRULE parameter.

Now that you have more detail, you know that there is a new parameter called ACCESSRULE that needs to be specificed in the manager parameter file. To find out more information on the ACCESSRULE parameter look at this link.

The defnition of this parameter is:

“Use ACCESSRULE to control connection access to the Manager process and the processes under its control. You can establish multiple rules by specifying multiple ACCESSRULE statements in the parameter file and control their priority. To establish priority, you can either list the rules in order from most important to least important,or you can explicitly set the priority of each rule with the PRI option.” 

This parameter is not a default configured parameter for the manager process. This is something that you will need to decided on and how to control access to your environments. The syntax for this parameter is as follows:

ACCESSRULE[, PROG program_name][, IPADDR address][, PRI rule][, login_ID]{,ALLOW | DENY}

There are multiple example of how to define an access rule in the Oracle GoldenGate 12.2.0.1 documentation. The example that I’m providing here is how I resolved the warning message in my Vbox test environment.

ACCESSRULE, PROG * ALLOW

With this access rule, you can now start the manager process without issue. You would be allowing access to all processes in the Oracle GoldenGate environment.

Enjoy!

-Bobby


Filed under: Golden Gate
Categories: DBA Blogs

Oracle GoldenGate 101 at IOUG 17 and GaOUG TechDays 17 within two month

Fri, 2017-03-31 08:00

Losts have changed for me since my last blog post. A new job and new responsibilities, which will include presenting more at conferences on the conference circuit and a few roadshows that are planned in the near future.

Back in the fall of 2016, when most of the Call for Papers open, I had a great idea for introducing people to Oracle GoldenGate. This idea is walk everyone through the progression of replication technology from CDC through Oracle GoldenGate, including some future road map items; that will ensure Oracle GoldenGate stays the number 1 replication tool on the market.

Over the next two months, I will be presenting this great idea/topic in a presentation called Oracle GoldenGate 101 – Replication from the beginning starting at IOUG Collaborate next week (April 2-6, 2017) and then at the best keep secret in the south – Georgia Oracle User Group (GaOUG) Tech Days 17 on May 9th-1th, 2017 (register here). This is going to be a session you will not want to miss, espeically if your organization uses Oracle GoldenGate for any architecture.

Additionally, the great guys who run the Georgia Oracle User Group (GaOUG) have done some amazing things since the planning of Tech Days 17 started. Just getting the key note speaker was a great addition to the already packed schedule of talks. This year, the keynote speaker is Maria Colgan, Master Product Manager for Oracle. She will be sharing some new and exciting items related to the Oracle Database and the direction Oracle is taking with new and exciting products.

GaOUG will also have dozens of Oracle ACE’s and ACE Director’s talking about topics accross multiple track (Database Development, Big Data, Applications, Middleteir). As an added bouns this year, GaOUG has added a half day Cloud track where you can learn all the items needed for your future Cloud deployments.

Hope to see everyone at both of these events, especially GaOUG in the coming months!

If you want to learn more about GaOUG Tech Days and the other impressive speakers, check out the blog hop posts below! What is a blog hop? It’s a group of bloggers who all get together to blog on a similar topic, giving you a lot of useful information from different perspectives, but in one stream. Enjoy your “hop” between blog posts and register for GaOUG Tech Days today!

Stewart Bryson
, RedPill Analytics
Jim Czuprynski, OnX Enterprise Solutions
Chris Lawless, Dbvisit Software
Danny Bryant, Accenture Enkitec Group
Eric Helmer, Mercury Technology Group


Filed under: GaOUG
Categories: DBA Blogs

Next Chapter … Time for a Change

Sat, 2017-01-21 20:06

It has been about three months since I’ve published a post. Well this time it has been a mix of work and decision making on my next chapter in my career that has kept me from posting. Hopefully, in the near future I will have more time to publish articles on topics I really want to discuss.

For the people close to me, many of you already know what this change is. I appreciate that you have kept it quite while all the details were worked out.

Before I get into what the next chapter of my career is, I have to say thanks to the management and team members at Accenture Enkitec Group (Enkitec/AEG). It has been a pleasure to work with everyone at Enkitec/AEG over the last couple of years. You truely are a great group of individuals and I will miss working daily with many of you. AEG is such a brain trust of people, it is almost hard to leave! I know that this will continue to be the trend at AEG.

Now what is the next chapter?

As of late January 2017, I have decided to leave Enkitec/AEG in pursuit of an opportunity that I feel is a good mix of my technical skills and allow me to grow in a challenging direction. Come Feburary 6th, 2017, I will be starting my new role as Senior Principal Product Manager on the GoldenGate team at Oracle. This position will allow me to work with a technology I’m very interested in and help shape the path forward for it. After all we know “cloud” is the next best thing right? How are you getting your data into or out of that “cloud”? Well, I will be part of the team that will help provide a path to do that. I’ll have to keep you posted on the technology as it evolves.

Now that you know where I’m going, just a few house keeping items to take care of. I’ve already said thanks to my friends at Enkitec/AEG. Without your support on many fronts, I wouldn’t be where I’m at as I proceed on this new chapter. Again, I greatly appreciate it.

For the community as a whole, many of you know that I’m an ACE Director (ACED) or at least was an ACED … LOL. Well, just like many of former ACEDs, when you go to Oracle you have to give it up. So, as of January 20, 2017, I resigned my ACED status and now listed as an ACE Alumni.

Blog site! I plan on keeping dbasolved.com up and running. I will be posting more article on GoldenGate here after starting with Oracle. So keep checking back to see if I’ve posted anything new. After all, GoldenGate is starting to become a hot topic!

I hope you will join me in my excitement on this new chapter of my career with Oracle and the GoldenGate team! I look forward to seeing many of you out and about within the community as I take on this new chapter.

Enjoy!

about.me:http://about.me/dbasolved


Filed under: Golden Gate
Categories: DBA Blogs

Fabric … Simple #GoldenGate Administration over SSH

Wed, 2016-10-05 10:45

 

For awhile now, I’ve been looking at different approaches for automating some of my GoldenGate monitor tasks. Yes, there are tools out there that provide a wide variety of ways for interaction and monitoring of Oracle GoldenGate. However, the problem with Oracle GoldenGate is that you have at least two parts of the equation when interacting or monitoring an environment, i.e. source and target. If you are so lucky to have a multi-master configuration, then you have to deal with multiple target sites. In the end, making administration and knowing what is going on a bit of a time-consuming process.

I’ve looked at different tools to try and solve this problem; mostly in Perl since I write a lot of that from time to time. Trying to get Perl to do network commands like SSH was time consuming and cumbersome when scripting it. This is when I started to look at Python. Initially I wasn’t sold on Python, but after spending some time to get familiar with the language, I actually like it; but it still was missing the SSH functionality I wanted, until recently.

As I was browsing the inter-web over the last week or so, I came across a blog post that compared deployment tools such as Chef, Puppet, Ansible, Fabric and a few others. The website provided pros and cons for each of the deployment options they were reviewing. Take a look and see what you may like to use for deployment options in your environments.

Out of the tools the website talked about, Fabric peaked my attention because it was a Python-based tool for streamlining SSH application deployments. I was thinking, if I could deploy applications with Fabric, then I should be able to do some basic monitoring with it as well; sending me down the path of solving basic Oracle GoldenGate monitoring from a single command line. Yea sounds like a good plan!

After spending some time, figuring out how to install Fabric and what exactly is a “fabfile”, I wrote my first monitoring script for Fabric!

What exactly does this monitoring script look like? Below is the monitoring script, also known as a “fabfile”. This script is written in Python syntax and sets up the environment and what functions should be called.

from fabric.api import *

env.hosts = [
'fred.acme.com',
'wilma.acme.com'
]

env.user = "oracle"
env.password = [ do not place in clear text ]

def info_all():
        with cd("$OGG_HOME"):
                run("echo info all | ./ggsci")

def ggstatus():
        info_all()

The environment that is going to be monitored are my test servers that I use for many of my tests (Fred and Wilma). Then I’m telling Fabric to use the “oracle” userid and password to login to each server. In this environment I have all my oracle passwords set the same so I only have to provide it once. After the environment is set, I define the functions that I want to perform. The first function is the info_all() function which is logging into GGSCI and providing me the output of the info all command. The second function is just a generic function that calls the first function.

With the “fabfile” created, I can now run a single command to access both servers (Fred & Wilma) and check on the status of the Oracle GoldenGate processes running on these servers. The command that I run is:

$ fab ggstatus -f ./fab_gg.py

This command is executing the Fabric executable “fab” followed by the function in the “fabfile” I want to execute. Then the “-f ./fab_gg.py” is the “fabfile” that I want to use during the execution. Upon execution, I will spool output on my STDOUT providing me the status of each Oracle GoldenGate environment for each server I’ve requested.

AMAC02P37LYG3QC:python bobby.l.curtis$ fab ggstatus -f ./fab_gg.py
[fred.acme.com] Executing task 'ggstatus'
[fred.acme.com] run: echo info all | ./ggsci
[fred.acme.com] out: The Oracle base has been set to /u01/app/oracle
[fred.acme.com] out: ====================================
[fred.acme.com] out: ORACLE_SID=src12c
[fred.acme.com] out: ORACLE_BASE=/u01/app/oracle
[fred.acme.com] out: ORACLE_HOME=/u01/app/oracle/product/12.1.0/dbhome_1
[fred.acme.com] out: OGG_HOME=/u01/app/oracle/product/12.2.0/oggcore_1
[fred.acme.com] out: JAVA_HOME=/home/oracle/Downloads/jdk1.8.0_71
[fred.acme.com] out: OGGSTUDIO_HOME=/u01/app/oracle/product/oggstudio/oggstudio
[fred.acme.com] out: OGGSTUDIO_HOME1=/u01/app/oracle/product/oggstudio/12.2.1.1.0/oggstudio
[fred.acme.com] out: ====================================
[fred.acme.com] out:
[fred.acme.com] out: Oracle GoldenGate Command Interpreter for Oracle
[fred.acme.com] out: Version 12.2.0.1.0 OGGCORE_12.2.0.1.0_PLATFORMS_151101.1925.2_FBO
[fred.acme.com] out: Linux, x64, 64bit (optimized), Oracle 12c on Nov 11 2015 03:53:23
[fred.acme.com] out: Operating system character set identified as UTF-8.
[fred.acme.com] out:
[fred.acme.com] out: Copyright (C) 1995, 2015, Oracle and/or its affiliates. All rights reserved.
[fred.acme.com] out:
[fred.acme.com] out:
[fred.acme.com] out:
[fred.acme.com] out: GGSCI (fred.acme.com) 1>
[fred.acme.com] out: Program     Status      Group       Lag at Chkpt  Time Since Chkpt
[fred.acme.com] out:
[fred.acme.com] out: MANAGER     RUNNING
[fred.acme.com] out: JAGENT      RUNNING
[fred.acme.com] out: EXTRACT     RUNNING     EGG12C      00:00:09      00:00:02
[fred.acme.com] out: Description 'Integrated Extract'
[fred.acme.com] out: EXTRACT     RUNNING     PGG12C      00:00:00      00:00:02
[fred.acme.com] out: Description 'Data Pump'
[fred.acme.com] out:
[fred.acme.com] out:
[fred.acme.com] out: GGSCI (fred.acme.com) 2>
[wilma.acme.com] Executing task 'ggstatus'
[wilma.acme.com] run: echo info all | ./ggsci
[wilma.acme.com] out: The Oracle base has been set to /opt/app/oracle
[wilma.acme.com] out: ====================================
[wilma.acme.com] out: ORACLE_SID=rmt12c
[wilma.acme.com] out: ORACLE_BASE=/opt/app/oracle
[wilma.acme.com] out: ORACLE_HOME=/opt/app/oracle/product/12.1.0.2/dbhome_1
[wilma.acme.com] out: OGG_HOME=/opt/app/oracle/product/12.2.0.0/oggcore_1
[wilma.acme.com] out: JAVA_HOME=/home/oracle/Downloads/jdk1.8.0_71
[wilma.acme.com] out: ODI_HOME=/opt/app/oracle/product/12.2.1.1.0/odi_home1
[wilma.acme.com] out: ====================================
[wilma.acme.com] out:
[wilma.acme.com] out: Oracle GoldenGate Command Interpreter for Oracle
[wilma.acme.com] out: Version 12.2.0.1.0 OGGCORE_12.2.0.1.0_PLATFORMS_151101.1925.2_FBO
[wilma.acme.com] out: Linux, x64, 64bit (optimized), Oracle 12c on Nov 11 2015 03:53:23
[wilma.acme.com] out: Operating system character set identified as UTF-8.
[wilma.acme.com] out:
[wilma.acme.com] out: Copyright (C) 1995, 2015, Oracle and/or its affiliates. All rights reserved.
[wilma.acme.com] out:
[wilma.acme.com] out:
[wilma.acme.com] out:
[wilma.acme.com] out: GGSCI (wilma.acme.com) 1>
[wilma.acme.com] out: Program     Status      Group       Lag at Chkpt  Time Since Chkpt
[wilma.acme.com] out:
[wilma.acme.com] out: MANAGER     RUNNING
[wilma.acme.com] out: JAGENT      RUNNING
[wilma.acme.com] out: REPLICAT    RUNNING     RGG12C      00:00:00      00:00:03
[wilma.acme.com] out:
[wilma.acme.com] out:
[wilma.acme.com] out: GGSCI (wilma.acme.com) 2>

Done.
Disconnecting from wilma.acme.com... done.
Disconnecting from fred.acme.com... done.

As you can tell, I get a scrolling output of both servers showing that the servers were logged into and changed directories to the $OGG_HOME. Then executed and “info all” command against GGSCI with the returning output. This makes for a quick and easy way to get the current status of all Oracle GoldenGate processes in an environment.

With this being an introduction to Fabric, I’m looking forward to seeing what else I can do with it. I’ll keep everyone posted on additional things I do may do with it. But for now, I encourage you to take a look at it and see if you can simplify some of your administration tasks with it.

Enjoy!!

about.me: http://about.me/dbasolved


Filed under: Dev Ops, Golden Gate
Categories: DBA Blogs

Oracle Open World 2016 – What GoldenGate sessions are there?

Wed, 2016-08-31 23:00

Well, it is that time of year again; the streets of San Francisco will be crowded with members of the Oracle community! As everyone in the Oracle community descends onto the bay area, there will be excitement about the things that will be announced this year. Sure a lot of it is going to be about “cloud” and what direction Oracle is taking with their “cloud” strategy. Besides, “cloud” there will be a lot of good things to take in as well. As I look through the online session catalog for 2016, I’m interested in the topics related to Oracle GoldenGate.

This year there appears to be a good number of Oracle GoldenGate sessions at Oracle Open World, to be specific there are 21 scheduled during the event. I have listed a few of the ones I think will be interesting and will make an attempt to attend; always check the session catalog because what I think is good you may not.

  • CON6632 – Oracle GoldenGate for Big Data
  • CON7318 – Getting Started with Oracle GoldenGate
  • CON6551 – New Oracle GoldenGate 12.3 Services Architecture (beta stuff)
  • CON6633 – Accelerate Cloud Onboarding with Oracle GoldenGate Cloud Service
  • CON6634 – Faster Design, Development and Deployment with Oracle GoldenGate Studio (should be updated from last year session)
  • CON6558 – Best Practice for High Availability and Performance Tuning for Oracle GoldenGate
  • UGF616 – Oracle GoldenGate and Apache Kafka: A Deep Dive into Real-Time Data Streaming
  • THT7817 – Real-Time and Batch Data Ingestion into Data Lake with Oracle GoldenGate Cloud Service
  • UGF5120 – Oracle GoldenGate and Baseball: Five Fundamentals Before Jumping to the Cloud

As we are within two weeks of Oracle Open World, I hope everyone is ready to go and looking forward to seeing others from the community.

Enjoy!

@dbasolved
http://about.me/dbasolved


Filed under: General
Categories: DBA Blogs

Digging into ADD SCHEMATRANDATA … what is #GoldenGate doing?

Fri, 2016-08-19 10:30

In my post on the differences between ADD TRANDATA and ADD SCHEMATRANDATA, I highlighted the differences between the two ways of adding supplemental logging to tables. One of the things I pointed out was that ADD SCHEMATRANDATA doesn’t add any supplemental log groups (SLG). Without SLGs then how does ADD SCHEMATRANDATA work? That was the question I was left with. So I started digging around to find the answer and I think you may be interested in this as well.

Tracing

In order to figure out what is actually going on within the database when ADD SCHEMATRANDATA is run, I had to trace the GoldenGate session. In order to do this, I first had to login to the database from GGSCI using “dblogin useridalias ggate”. After logging in, I had to identify the session and then setup tracing. This was done with the following scripts:

—SQL to identify the GGate session
select 'exec dbms_monitor.session_trace_enable(session_id=>'||sid||', serial_num=>'||serial#||', binds=>true, waits=true)' 
from v$session where username = 'GGATE';

—Start tracing of the session
exec dbms_monitor.session_trace_enable(session_id=>156, serial_num=>15799, binds=>true, waits=>true);
—Disable tracing after done
exec dbms_monitor.session_trace_disable(session_id=>156, serial_num=>15799);

Now with tracing enabled for the session, I’m able to trace the GoldenGate session from within GGSCI. At this time, I’m able to run DELETE SCHEMATRANDATA and ADD SCHEMATRANDATA against the schema I want to add the logging to.

All the last step of tracing, I need to disable the tracing (script above) of the GoldenGate user and identify the trace file. By default the trace file should be in $ORACLE_BASE/diag/rdbms/<db>/<db>/trace. In this directory, you need to identify the trace files for the session that was traced.

oracle >ls -ltr src12c_ora_23267.*
-rw-r-----. 1 oracle oracle   2788 Aug 15 16:09 src12c_ora_23267.trm
-rw-r-----. 1 oracle oracle 300531 Aug 15 16:09 src12c_ora_23267.trc

After you know the trace file, you can perform a TKPROF on the trace file.

oracle > tkprof src12c_ora_23267.trc src12c_ora_23267.tkprofs

This is generate a tkprofs file that will show specifics for the sessions, such as the SQL that is ran. This is what I’m more interested in, what SQL is ran with ADD SCHEMATRANDATA is called.

Quick Review of TKProf file

I’m not going to go into all the details that are in a tkprof generated file, but I took a look through the file trying to figure out what is going on when running the ADD SCHEMATRANDATA. As I was searching the file, I found a reference to LOGMNR$ALWAYS_SUPLOG_COLUMNS.

If you have kept up with the versions of Oracle GoldenGate, you will know that this is a reference to LogMiner and that Oracle is integrating the Oracle GoldenGate processes with it. This is also a hint to where to look, a.k.a Oracle Streams. As I continued to look through the tkprof file, I found a few references to a streams package – DBMS_CAPTURE_ADM; along with sql statements making calls to views like STREAMS$_PREPARE_DDL.

BEGIN sys.dbms_capture_adm.PREPARE_SCHEMA_INSTANTIATION('soe','ALLKEYS_OFF');
  END;
BEGIN sys.dbms_capture_adm.ABORT_SCHEMA_INSTANTIATION('soe'); END;

At this point, it is safe to say, that Oracle is merging Oracle GoldenGate into Oracle Streams.

Review of Packages

Note: These are Oracle packages that are encrypted in the database. You can use UnwrapIt to view if needed. Output here is only for teaching purposes.

Now that I’m armed with what package the ADD SCHEMATRANDATA is calling, I can drill into what exactly is happening.

In taking a look at the DBMS_CAPTURE_ADM package, I wanted to look at the PREPARE_SCHEMA_INSTANTIATION procedure. I can quickly see that the procedure takes the schema_name and that defaults will be used for supplemental logging and container.

PROCEDURE PREPARE_SCHEMA_INSTANTIATION( 
 SCHEMA_NAME               IN VARCHAR2, 
 SUPPLEMENTAL_LOGGING      IN VARCHAR2 DEFAULT 'KEYS', 
 CONTAINER                 IN VARCHAR2 DEFAULT 'CURRENT') 
 IS 
 BEGIN 
 DBMS_CAPTURE_ADM_IVK.PREPARE_SCHEMA_INST_IVK( 
 SCHEMA_NAME => SCHEMA_NAME,  
 SUPPLEMENTAL_LOGGING => SUPPLEMENTAL_LOGGING, 
 CONTAINER => CONTAINER); 
 END;

After the schema is passed to the procedure, this procedure passes all three of the parameters to another package and procedure for execution. This package is DBMS_CAPTURE_ADM_IVK.PREPARE_SCHEMA_INST_IVK.

PROCEDURE PREPARE_SCHEMA_INST_IVK(
SCHEMA_NAME               IN VARCHAR2,
SUPPLEMENTAL_LOGGING      IN VARCHAR2 DEFAULT 'KEYS',
CONTAINER                 IN VARCHAR2 DEFAULT 'CURRENT')
IS
SUPP_LOG_LEVEL      BINARY_INTEGER;    
SYNCHRONIZATION     VARCHAR2(4) := 'LOCK';
CANON_CONTAINER     DBMS_ID;
BEGIN
  DBMS_LOGREP_UTIL.WRITE_TRACE('prepare_schema_inst_ivk()+ container: ' ||
  CONTAINER, DBMS_LOGREP_UTIL.TRACE_FLAG_PUBLISH);
  COMMIT;

  SUPP_LOG_LEVEL := DBMS_CAPTURE_ADM_IVK.SUPP_LOG_LEVEL_CHECK(SUPPLEMENTAL_LOGGING, 
  IF DBMS_XSTREAM_GG_ADM.SYNCHRONIZATION IS NOT NULL 
  THEN
         SYNCHRONIZATION := UPPER(DBMS_XSTREAM_GG_ADM.SYNCHRONIZATION);
  END IF;

  IF (SYNCHRONIZATION != 'NONE' AND SYNCHRONIZATION != 'LOCK' AND
           SYNCHRONIZATION != 'WAIT') 
  THEN
           DBMS_LOGREP_UTIL.RAISE_SYSTEM_ERROR(
           DBMS_STREAMS_ADM.INVALID_PARAMETER_NUM,
           DBMS_LOGREP_UTIL.GET_PROC_USE_CONTEXT, 'SYNCHRONIZATION'); 
  END IF;

  DBMS_UTILITY.CANONICALIZE(CONTAINER, CANON_CONTAINER, 30);   
  IF CANON_CONTAINER = 'CURRENT' 
  THEN
     DBMS_LOGREP_UTIL.WRITE_TRACE('prepare schema objects for current' ||
            ' container: ', DBMS_LOGREP_UTIL.TRACE_FLAG_PUBLISH);
     DBMS_CAPTURE_ADM_INTERNAL.PREPARE_SCHEMA_INSTANTIATION(
        SCHEMA_NAME, SYS_CONTEXT('USERENV','_USER'), SUPP_LOG_LEVEL,
        SYNCHRONIZATION, CANON_CONTAINER);
  ELSE
     DBMS_CAPTURE_ADM_INTERNAL.PREPARE_SCHEMA_INST_CDB(
            SCHEMA_NAME, CANON_CONTAINER,
            SYS_CONTEXT('USERENV','_USER'), SUPP_LOG_LEVEL,
           SYNCHRONIZATION);
  END IF;

  COMMIT;
  END;

In this procedure, you will notice that the input is all three of the parameters that are passed from the PREPARE_SCHEMA_INSTANTIATION procedure. Then this procedure precedes to check the supplemental logging and attempts to sync the tables with the DBMS_XSTREAM_GG_ADM package. After the tables have been synced, then the procedure calls another package and procedure to prepare (DBMS_CAPTURE_ADM_INTERNAL.PREPARE_SCHEMA_INSTANTIATION) the tables with the required supplemental logging level.

When looking at this procedure, it takes in five different parameters; two of which are defaults. After the procedure sets up tracing, it checks to see if the calling procedure is specifying GoldenGate. Once everything is confirmed and synchronization is complete, then the procedure grabs the CURRENT_SCN, waits for any inflight transactions and prepares the schema tables before exiting the package.

PROCEDURE PREPARE_SCHEMA_INSTANTIATION(
SCHEMA_NAME               IN VARCHAR2,
CANON_INVOKER             IN VARCHAR2,
SUPP_LOG_LEVEL            IN BINARY_INTEGER,
SYNCHRONIZATION           IN VARCHAR2 DEFAULT 'LOCK',
CONTAINER                 IN VARCHAR2 DEFAULT 'CURRENT')
IS
CANON_SCHEMA  DBMS_ID;
IDX           NUMBER :=0;
NAME_ARRAY    DBMS_UTILITY.NAME_ARRAY;
OWNER_ARRAY   DBMS_UTILITY.NAME_ARRAY;
CURRENT_SCN   NUMBER;
WFIT          BOOLEAN := FALSE;
RUN_TKLRWT1   BOOLEAN := FALSE;
SUPPLOG       BINARY_INTEGER := SUPP_LOG_LEVEL;
TEST_EVENT_LVL   BINARY_INTEGER := 0;
BEGIN
 DBMS_LOGREP_UTIL.WRITE_TRACE(
 'dbms_capture_adm_internal.prepare_schema_instantiation()+', DBMS_LOGREP_UTIL.TRACE_FLAG_PUBLISH);
 DBMS_LOGREP_UTIL.WRITE_TRACE('schema_name='||SCHEMA_NAME,   DBMS_LOGREP_UTIL.TRACE_FLAG_PUBLISH);
  DBMS_LOGREP_UTIL.WRITE_TRACE('supp_log_level='||SUPP_LOG_LEVEL, DBMS_LOGREP_UTIL.TRACE_FLAG_PUBLISH); 
 DBMS_UTILITY.CANONICALIZE(SCHEMA_NAME, CANON_SCHEMA, 30);  

 IF ((SUPP_LOG_LEVEL = DBMS_STREAMS_DECL.SUPPLOG_ALLKEYS_ON) OR (SUPP_LOG_LEVEL = DBMS_STREAMS_DECL.SUPPLOG_ALLKEYS_OFF)) 
 THEN
    STORE_PREPARE_INFO(CANON_SCHEMA, SUPP_LOG_LEVEL);
    RETURN;
 END IF;
    DBMS_LOGREP_UTIL.WRITE_TRACE('dbms_capture_adm_internal.prepare_schema_instantiation()+ before read_ev', DBMS_LOGREP_UTIL.TRACE_FLAG_PUBLISH);
    DBMS_SYSTEM.READ_EV(DBMS_LOGREP_UTIL.EXT_TRACE_EV, TEST_EVENT_LVL);

    IF (DBMS_XSTREAM_GG_ADM.IS_GOLDENGATE AND TEST_EVENT_LVL = 0) 
    THEN
     SUPPLOG := 0;
    END IF;
   DBMS_LOGREP_UTIL.WRITE_TRACE('dbms_capture_adm_internal.prepare_schema_instantiation()+ before prepare_ddl',DBMS_LOGREP_UTIL.TRACE_FLAG_PUBLISH);
    PREPARE_DDL(CANON_SCHEMA, SUPPLOG);
    DBMS_LOGREP_UTIL.WRITE_TRACE('dbms_capture_adm_internal.prepare_schema_instantiation()+ before prepare_schema_tables',DBMS_LOGREP_UTIL.TRACE_FLAG_PUBLISH);
	RUN_TKLRWT1 := RUN_WFIT_TEST;

    IF (UPPER(SYNCHRONIZATION) = 'NONE') 
    THEN
		PREPARE_SCHEMA_TABLES(CANON_SCHEMA, FALSE, CONTAINER);
    ELSIF (UPPER(SYNCHRONIZATION) = 'LOCK') 
    THEN
		PREPARE_SCHEMA_TABLES(CANON_SCHEMA, TRUE, CONTAINER);
    ELSIF (UPPER(SYNCHRONIZATION) = 'WAIT') 
    THEN
		WFIT := TRUE;
 		CURRENT_SCN  := DBMS_FLASHBACK.GET_SYSTEM_CHANGE_NUMBER;
        DBMS_CAPTURE_ADM_INTERNAL.WAIT_FOR_INFLIGHT_TXN(CURRENT_SCN, 'Prepare_schema_instantiation');
        PREPARE_SCHEMA_TABLES(CANON_SCHEMA, FALSE, CONTAINER);
    END IF;
        DONE_WFIT_TEST(WFIT, RUN_TKLRWT1);
    END;

Up to this point, the last three packages have been preparing the tables. Seems like a lot of preparing to get tables set for supplemental logging. Well the last call in the previous package called the procedure to prepare the tables. When I look at this procedure, I see that there is a call to a PREPARE_TABLE_INST procedure (getting a bit lazy on copying the procedure – don’t wan to put to much since this is Oracle’s code).

FOR I IN 1..IDX LOOP
BEGIN
   DBMS_LOGREP_UTIL.WRITE_TRACE('prepare_schema_tables():' || CANON_OWNER_ARRAY(I) || '.' || CANON_NAME_ARRAY(I), DBMS_LOGREP_UTIL.TRACE_FLAG_PUBLISH);
   PREPARE_TABLE_INST(CANON_OWNER_ARRAY(I), CANON_NAME_ARRAY(I),DBMS_STREAMS_DECL.SUPPLOG_NONE, LOCKING, CONTAINER);
EXCEPTION WHEN OTHERS THEN
   DBMS_LOGREP_UTIL.DUMP_TRACE('error:dbms_capture_adm_internal.prepare_table_inst('||'owner='||CANON_OWNER_ARRAY(I)||' name='||CANON_NAME_ARRAY(I));
END;

Now, when I go an look at the PREPARE_TABLE_INST procedure, I see that the procedure is calling a C package called “knlcpreptabinst” to set the supplemental logging on the tables associated with the schema.

PROCEDURE PREPARE_TABLE_INST(
CANON_OWNER                 IN VARCHAR2,
CANON_TABLE_NAME            IN VARCHAR2,
SUPPLEMENTAL_LOGGING_LEVEL  IN BINARY_INTEGER,
LOCKING                     IN BOOLEAN,
CONTAINER                   IN VARCHAR2) 
IS
EXTERNAL
 NAME "knlcpreptabinst"
 LIBRARY DBMS_LOGREP_LIB
 WITH CONTEXT
 PARAMETERS(CONTEXT,    
      CANON_OWNER       OCISTRING, CANON_OWNER       INDICATOR SB2,
      CANON_TABLE_NAME  OCISTRING, CANON_TABLE_NAME  INDICATOR SB2,
      SUPPLEMENTAL_LOGGING_LEVEL UB4,
      LOCKING           UB2,       LOCKING           INDICATOR SB2,
      CONTAINER         OCISTRING, CONTAINER         INDICATOR SB2)
LANGUAGE C;

Without drilling down into the C package, I cannot see exactly how the supplemental logging is added or where it is exactly stored in the database. I can only assume (we know what that means … lol), that it is in the data dictionary some where.

Looking for tables or views that may shed some light on this as well, I’ve found LOGMNR$SCHEMA_ALLKEY_SUPLOG that will show you the schema, if all keys are in supplemental log mode and if no validated pks are allowed. The following query is what I used to extract information about the SOE schema:

select * from LOGMNR$SCHEMA_ALLKEY_SUPLOG
where allkey_suplog = 'YES';

 

Output is as follows:

SCHEMA_NAME     ALL ALL
--------------- --- ---
SOE             YES NO

In drilling down further, after have a friend of mine pointed out a function to me (follow him on twitter -> @resetlogs). You can get down to the table level on supplemental logging when using ADD SCHEMATRANDATA. There is a log miner function that has to be called when using SQL to pull the correct information. This function is similar named to the table I referenced above … LOGMNR$ALWAYS_SUPLOG_COLUMNS.

This function takes two parameters. The first is the schema that holds the objects and the second is the table name. So in the following example, I can see that the ORDERS table of the SOE schema has supplemental logging added.

select * from table(logmnr$always_suplog_columns('SOE','ORDERS'));

OWNER           TABLE_NAME                     COLUMN_NAME                        INTCOL     SEGCOL    USERCOL
--------------- ------------------------------ ------------------------------ ---------- ---------- ----------
SOE             ORDERS                         ORDER_ID                                1          1          1
SOE             ORDERS                         ORDER_TOTAL                             6          6          6
SOE             ORDERS                         COST_OF_DELIVERY                       11         11         11
SOE             ORDERS                         DELIVERY_ADDRESS_ID                    13         13         13
SOE             ORDERS                         ORDER_DATE                              2          2          2
SOE             ORDERS                         CUSTOMER_CLASS                         14         14         14
SOE             ORDERS                         CUSTOMER_ID                             4          4          4
SOE             ORDERS                         ORDER_STATUS                            5          5          5
SOE             ORDERS                         PROMOTION_ID                            8          8          8
SOE             ORDERS                         ORDER_MODE                              3          3          3
SOE             ORDERS                         SALES_REP_ID                            7          7          7
SOE             ORDERS                         WAREHOUSE_ID                            9          9          9
SOE             ORDERS                         DELIVERY_TYPE                          10         10         10
SOE             ORDERS                         WAIT_TILL_ALL_AVAILABLE                12         12         12
SOE             ORDERS                         CARD_ID                                15         15         15
SOE             ORDERS                         INVOICE_ADDRESS_ID                     16         16         16

 

I know this has been a long post, but hopefully, I’ve been able to somewhat show how the ADD SCHEMATRANDATA command within GGSCI works and where you can see if supplemental logging is turned on for the selected schema. As I was trying to dig to the root of this issue, I found it interesting that so many packages are involved with setting the supplemental logging on a schema/tables; while identifying if it is enabled is not as easy as ADD TRANDATA. Where when you use ADD TRANDATA the tables are easily identified and can quickly see that the supplemental log groups have been added.

As Oracle GoldenGate for Oracle moves to a more integrated approach to replication, I think more items will be tied to the log miner and streams architecture.

Enjoy!!

@dbasolved
http://about.me/dbasolved


Filed under: Golden Gate
Categories: DBA Blogs

rlwrap with #GoldenGate GGSCI

Tue, 2016-08-16 10:45

Since I published posts on how to retrieve command history within GGSCI using the Oracle provided commands of “history” and “fc”, I was pinged a couple of times by some friends asking about “rlwrap” usage with GoldenGate. The answer is a simple yes, rlwrap can be used with Oracle GoldenGate.

What exactly is “rlwrap”?

According to the readme file at http://utopia.knoware.nl/~hlub/uck/rlwrap/#rlwrap, rlwrap is a ‘read-one wrapper’. Basically, it allows for the editing of keyboard input for any command.

How to install?

There are two ways to install “rlwrap”. The first way is manually, which requires you to run the configure and make commands; however, the easiest way I’ve found is by using a yum repository. Tim Hall (a.k.a Oracle-Base) documents this process quite effortlessly over at his blog. Thanks Tim!

Usage?

After you have “rlwrap” installed, you simply have to create an alias in your profile to use it. You can use it from the command line as well; however, the alias approach ensure that it is executed every time you run ggsci. The below example is what I have running in my .bash_profile for the Oracle user.

alias ggsci='rlwrap $OGG_HOME/ggsci'

The next time I login to use ggsci, rlwrap will automatically be ran and then I will be able to scroll through the commands I typed while in ggsci. Another nice thing about “rlwrap” is that when I logout of ggsci, and then back in, my command history is available still.

Although the “history” and “fc” commands are handy, it would be nice to see Oracle include “rlwrap” into the core product of Oracle GoldenGate …

Enjoy!

@dbasolved
http://about.me/dbasolved


Filed under: Golden Gate
Categories: DBA Blogs

Pages