Wpomatic WP-o-Matic

Comments and changes to this ticket

• You flagged this item as spam.
  

  KBruno November 5th, 2008 @ 08:36 PM

  Sorry, I posted my reply to you in the wrong spot. Anyway...

  This is not fix for whatever it is in wp-o-matic that causes this to happen (I haven't found anything that even claims to have an answer to exactly why this happens), but it will stop it from happening.

  In /wp-includes/post.php there are 2 places you'll find "$suffix = 2;". Replace both with "return 0;". That only looks at the post_name (Title) and nothing else so it won't help with morphing feeds or the like. It will also work on all posts and not just on wp-o-matic imports. I do know if you manually write a post with the same title as a previous post it'll go into Drafts until you change the title in some way but I think imports just get tossed. I know I lose a few updates on some feeds but that's well worth not having to delete all the duplicates and not having the server all but shut down once in awhile.

  Hope that works for you. :-)

• You flagged this item as spam.
  

  trapmuzik November 20th, 2008 @ 11:56 AM

  I have the same problem. It doesn't seem to happen all the time but all of my feeds have the exact same problem. I tried to add some debugging code to the script to catch why it was inserting the same post multiple times. Still haven't find out why. This only started happening after my host changed servers and upgraded php versions. I dont see why that would matter. The problem may be with the simplepie class.

• You flagged this item as spam.
  

  trapmuzik November 22nd, 2008 @ 02:09 AM

  I think I found out what the problem is with multiple posts. After reviewing some of the wp-o-matic logs I noticed that the cron job was running 2-3 times at the exact same time which means the wp-o-matic script is being executed multiple times at the exact same time. So my theory is you have two or three instances of the script running simultaneously and thus inserting the same content sometimes 2,3, or 4 times without being caught by the duplicate filter function.

  So then I was trying to figure out why the cron job would be running multiple times if I only had one cron job to execute the cron.php script. I checked the crontab for my user and it was only there once. I then deleted the cron job and I was still getting cron job messages in the logs. At this point I figured the script itself was doing its on cron job. I had the box checked to do "Unix Cron" in the options but it was still running a cron job even though I had none. I toggled the option for Unix Cron and was still getting cron job messages. But since I deleted my cron job from crontab I am not getting any duplicates so far.

• You flagged this item as spam.
  

  johnspice December 2nd, 2008 @ 02:18 PM

  use this modified version which I made, most features work fine for now!

  • wp-o-matic_Working_Version_wordpress_plugin.zip 141.6 KB

• You flagged this item as spam.
  

  elindbloom December 3rd, 2008 @ 12:12 AM

  I have a problem with only one feed that seems to pull the same story every time a cron runs in a given day. It is also the only feed that comes through feedburner. Is there an issue with pulling through feedburner?

• You flagged this item as spam.
  

  snewe December 3rd, 2008 @ 06:27 PM

  I also have this problem with 1.0RC4-6. I see that the hash field in _wpo_campain_post is used to check for duplicates. This does not work for my setup with 4 separate campaigns with unique feeds for each. Couldn't a simple check for an existing permalink do the trick?

• You flagged this item as spam.
  

  snewe December 3rd, 2008 @ 06:50 PM

  Nevermind...I had non-unique feeds. Also, "johnspice" appears to be spam. Be careful of that zip file.

• You flagged this item as spam.
  

  low-R December 11th, 2008 @ 06:15 AM

  • Tag set to “duplicate post, solution”

  I had the same problem. To correct that just had this function at the end of the file wpomatic.php

  
  // duplicate posts
  
  $wpdb->query('
  delete bad_rows.*
  from wp_posts as bad_rows
  inner join (
  select post_title, MIN(id) as min_id
  from wp_posts
  group by post_title
  having count(*) > 1
  ) as good_rows on good_rows.post_title = bad_rows.post_title
  and good_rows.min_id <> bad_rows.id;
  ');
  

• You flagged this item as spam.
  

  KBruno December 12th, 2008 @ 03:25 PM

  low-R, that works to remove duplicate titled posts as the return 0 in post.php works to prevent duplicate titled posts.

  The real trick was to allow duplicate titles if the post content is different (common in many feeds). I got that licked in post.php by comparing the content when the titles are the same. I replaced the $suffix = 2; with this:

          $post_content_check = $wpdb->get_var($wpdb->prepare("SELECT post_content FROM $wpdb->posts WHERE post_name = %s AND post_type = %s AND ID != %d AND post_parent = %d LIMIT 1", $post_name, $post_type, $post_ID, $post_parent));
  
          $p_c_c = ltrim($post_content_check);
          $p_c = ltrim($post_content);
          $p_c_c = rtrim($p_c_c);
          $p_c = rtrim($p_c);
          $p_c_c = preg_replace('/\s\s+/', '', $p_c_c);
          $p_c = preg_replace('/\s\s+/', '', $p_c);
          $p_c_c = sha1($p_c_c);
          $p_c = sha1($p_c);
  
          if ($p_c_c == $p_c) {
  
              return 0;
  
          } else {
  
                  $suffix = 2;
  
          }
  
  

  Works well but in this case though I don't think is possible to eliminate true duplicates all together. It seems that no matter what the cron settings, wpom can be called more than once at the same time & if the first instance hasn't inserted a post into the db when the second instance looks for it you've got a duplicate.

  If there were a way to prevent wpom from being called while it's still running (no reason for that that I can see) I think it would solve the problem completely. Anyone know how to do that?

• You flagged this item as spam.
  

  jibrail April 12th, 2009 @ 10:32 AM

  Same Problem here... Any fix for it?

• You flagged this item as spam.
  

  KBruno April 12th, 2009 @ 03:26 PM

  jibrail, I haven't seen anything new on this at all. I was hoping Guillermo Rauch or anyone who knows more than me (and that's not alot) about how Wordpress plugins are called would chime in but there's been nothing since my last post.

  I'll ask again; does anyone know how to prevent WPOMatic from being initiated when it's already running?

• You flagged this item as spam.
  

  Ron April 12th, 2009 @ 03:30 PM

  I don't think you'll find a way, its a race condition that exists at the design level, not the code level.

  I'm using the cron job to periodically update my feeds and that works without getting duplicates.

• You flagged this item as spam.
  

  vboogieman May 6th, 2009 @ 09:15 PM

  Would love to see an update to stop the duplicate posts :)

  I have to go through my posts multiple times a day to remove the duplicates

• You flagged this item as spam.
  

  thecolor June 15th, 2009 @ 04:41 PM

  As of the most recent upgrade to WP, it seems the $suffix = 2; changed to return 0; no longer works.

  Sadly, all my gReader posts are coming through about 7 times or more. Thoughts?

• You flagged this item as spam.
  

  Eric August 11th, 2009 @ 12:14 AM

  hey everyone, so is there any solution or fix on this? seems there is no reply whatsoever from the developer etc?
  i keep getting the same post multiple times on my blog, up to 10 or more times. so if i am fetching 5 posts from the feed, each post will generate up to 10 multiple posts of the exact same content on my blog, meaning i will get all these 50 posts of same content and i have to remove all of them manually! this will defeat the purpose of the automation. if this isn't fixed, i do not see the purpose of this script anymore. i am using the wp to handle the cron job, as i am not using any unix or web cron jobs.

  PLS HELP!!! i see this thread originate so many months back, but there is no real solution??? PLS PLS PLS... anyone ple provide a solution pls.......

• 

  Ricardo Rauch April 15th, 2010 @ 08:50 AM

  • Assigned user changed from “Guillermo Rauch” to “Ricardo Rauch”

  [bulk edit]