oracle 10g ora-07445 rac环境下一个案例,主要排查以下错误日志:tomcat日志、数据库alter日志、数据表日志错误、数据库转储文件
oracle 10gora-074451. 数据库使用的软件及其版本
序号
名称
内容
1
操作系统
centos release 5.6 (final) x86-64
2
数据库版本
oracle database 10g release 2
3
数据库安装包:
补丁包:
10201_database_linux_x86_64.cpio.gz
p6810189_10204_linux-x86-64.zip
4
oembug解决补丁
p8350262_10204_generic.zip
2. 数据库结构
3. 现象现在出现的报表系统goldengate同步进程异常中断,还有修改部分product表内容rollback。数据库后台报严重的错误。
主要排查以下错误日志:tomcat日志、数据库alter日志、数据表日志错误、数据库转储文件、goldengate进程、goldengate同步日志。
3.1 tomcat日志错误jdbc版本:ojdbc14-10.2.0.4.0.jar
2012-05-14 16:15:36,755 [http-8083-3] error net.sf.hibernate.transaction.jdbctransaction - rollback failed
java.sql.sqlexception: no more data to read from socket
at oracle.jdbc.driver.databaseerror.throwsqlexception(databaseerror.java:112) ~[ojdbc14.jar:oracle jdbc driver version - 10.2.0.4.0]
at oracle.jdbc.driver.databaseerror.throwsqlexception(databaseerror.java:146) ~[ojdbc14.jar:oracle jdbc driver version - 10.2.0.4.0]
3.2 数据库alter日志
mon may 14 14:37:33 2012
errors in file /oracle/ora10g/admin/jscn/udump/jscn2_ora_28001.trc:
ora-07445: exception encountered: core dump [_intel_fast_memcpy.a()+10] [sigsegv] [address not mapped to object] [0x2b09d44be000] [] []
mon may 14 14:48:14 2012
process 0x0x2d8367dc0 appears to be hung while dumping
current time = 485605312, process death time = 485541208 interval = 60000
从上面的日志可以看出进程hung住,,让我查看/oracle/ora10g/admin/jscn/udump/jscn2_ora_28001.trc转存文件
3.3 查看转存文件/oracle/ora10g/admin/jscn/udump/jscn2_ora_28001.trc转存文件比较长,大概有5m,我们就看关键部分
exception signal: 11 (sigsegv), code: 1 (address not mapped to object), addr: 0x2b09d44be000, pc: [0x2b09d26ee608, _intel_fast_memcpy.a()+10]
*** 2012-05-14 14:37:33.145
ksedmp: internal or fatal error
ora-07445: exception encountered: core dump [_intel_fast_memcpy.a()+10] [sigsegv] [address not mapped to object] [0x2b09d44be000] [] []
current sql statement for this session:
update product set type=:1, ship_type=:2, stock_warning2=:3, stock_warning=:4, store_id=:5, product_name=:6, unit_price=:7, list_price=:8, quantity=:9, brand=:10, model=:11, spec=:12, unit=:13, manufactory=:14, producingarea=:15, reseller=:16, supplier_id=:17, create_time=:18, update_time=:19, description=:20, big_picture=:21, small_picture=:22, center_picture=:23, pop_picture=:24, pop_picture1=:25, pop_picture2=:26, pop_picture3=:27, pop_picture4=:28, pop_picture5=:29, note=:30, template_id=:31, checkstatus=:32, check_man=:33, check_time=:34, status=:35, searchkey=:36, main_category=:37, product_no=:38, sid=:39, displayid=:40, sortindex=:41, corpid=:42, linkurl=:43, spec_name1=:44, spec_name2=:45, spec_value1=:46, spec_value2=:47, pop_type=:48, tag=:49, visitcount=:50, starttime=:51, endtime=:52, refer_price=:53, base_price=:54, parent_id=:55, isconfig=:56, short_name=:57, custom_image=:58, app_usercount=:59, app_totalcount=:60, buy_totalcount=:61, weight=:62, integral=:63, integral_count=:64, is_userprice=:65, short_name_color=:66, short_name_bold=:67, p_color=:68, p_color_pic=:69, charges_flag=:70, ext_field1=:71, ext_field2=:72, extfileld3=:73, extfileld4=:74, ext_fileld5=:75, ext_fileld6=:76, ext_fileld7=:77, ext_fileld8=:78, ext_fileld9=:79, ext_fileld10=:80, ext_fileld11=:81, ext_fileld12=:82, ext_fileld13=:83, ext_fileld14=:84, ext_fileld15=:85, ext_fileld16=:86, ext_fileld17=:87, ext_fileld18=:88, ext_fileld19=:89, ext_fileld20=:90, esalesno=:91, classtype=:92, game_type=:93, each_buy_count=:94, limit_buy_count=:95, start_limit_time=:96, end_limit_time=:97, channel_flag=:98, ydprice=:99, tqyd_days=:100, scope=:101, vd_id1=:102, vd_id2=:103, vd_id3=:104, vd_id4=:105, vd_id5=:106, auto_gtcard=:107, tv_note=:108, mobile_note=:109, is_yyproduct=:110, ext_fileld41=:111, warranty=:112, agent_id=:113, min_buynum=:114, model_no=:115, tp_spec_value1=:116, tp_spec_value2=:117 where product_id=:118
----- call stack trace -----
ora-07445这个是个内部错误,查看oracle官方文档解释
ora-07445:exception encountered: core dump [string] [string] [string] [string]
[string][string]
cause: an os exception occurred which should resultin the creation of a core file.
this is aninternal error.
action: contact your customer support representative.